Build scalable and reliable data pipelines that collect, transform, load and curate data from internal systems
Augment data platform with data pipelines from external systems.
Ensure high data quality for pipelines you build and make them auditable
Drive data systems to be as near real-time as possible
Support design and deployment of distributed data store that will be central source of truth across the organization
Build data connections to company's internal IT systems
Develop, customize, configure self service tools that help our data consumers to extract and analyze data from our massive internal data store
Evaluate new technologies and build prototypes for continuous improvements in data engineering.

Requirements

5+ years of work experience in relevant field (Data Engineer, DWH Engineer, Software Engineer, etc)
Experience with data-lake and data-warehousing technologies and relevant data modeling best practices (Presto, Athena, Glue, etc)
Proficiency in at least one of the main programming languages used: Python and Scala. Additional programming languages expertise is a big plus!
Experience building data pipelines/ETL in Airflow, and familiarity with software design principles.
Excellent SQL and data manipulation skills using common frameworks like Spark/PySpark, or similar.
Expertise in Apache Spark, or similar Big Data technologies, with a proven record of processing high volumes and velocity of datasets.
Experience with business requirements gathering for data sourcing.
Bonus - Kafka and other streaming technologies like Apache Flink.

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data pipelinesdata warehousingdata modelingPythonScalaETLSQLApache SparkAirflowdata manipulation

Soft Skills

data quality assuranceauditingdesigndeploymentcontinuous improvementbusiness requirements gathering