Tech Stack
AirflowAngularAnsibleAWSCloudDistributed SystemsHadoopJavaJenkinsMicroservicesPySparkPythonReactSparkSQLTerraform
About the role
- Develop best practices and make architectural choices to improve data processing and analytics pipelines
- Collaborate with software engineers and support solutions to modern engineering problems
- Build high-performance, fault-tolerant, and scalable distributed systems that can handle massive scale
- Diagnose issues in distributed systems, develop and document technical solutions, and sequence work for iterative deliveries
- Participate in design reviews and review code developed by other developers
- Automate cloud infrastructure, services, and observability
- Develop CI/CD pipelines and testing automation
- Establish and uphold best engineering practices through code and design reviews and improved tools
- Mentor junior engineers and create a culture of trust, respect, and inclusion
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent training, fellowship, or work experience
- 5+ years of relevant industry experience in large back-end distributed systems and cloud computing
- Solid overall programming skills, able to write modular, maintainable code in Java, Python, and SQL
- Expert understanding of SQL, dimensional modeling, and at least one relational database, including solid contributions to ERD
- Good understanding of front-end frameworks like ReactJS/AngularJS
- Solid proficiency with automation frameworks/tools like Git, Jenkins, Ansible, and CloudFormation (or Terraform)
- Solid proficiency with containers and infrastructure-as-code fundamentals
- Solid proficiency with Amazon Web Services
- Good understanding of Data Engineering and related frameworks
- Familiarity with MVC, SOA, and RESTful web services
- Familiarity with non-functional testing such as load, performance, and resiliency testing
- Experience working in an Agile environment using test-driven methodologies
- Commitment to deepen knowledge of how to improve our existing applications
- Enthusiasm for modern technologies, complex problems, and building things
- Preferred: Experience with Spark/PySpark, Iceberg, Presto, Hive or Snowflake, Hadoop 2.0 and its ecosystem, Airflow, and microservices-based architecture