Develop, optimize, and improve data systems such as ETL data, pipeline, storage, and entity resolution
Build infrastructure and data automation pipelines for the ingestion, processing, and loading of data from various sources
Automate and integrate new components into the data pipeline
Collaborate with stakeholders and data science teams to develop data products
Design, build, and maintain scalable data pipelines and infrastructure for machine learning workflows
Create unit and stress test components to monitor technical performance
Develop data analysis tools to provide data insights and capture key metrics
Follow best practices for data governance, quality, cleansing, and other ETL-related activities

Requirements

7+ years of development experience in data engineering within a production environment (internships and academic settings excluded)
Proven experience working with large datasets exceeding 100M+ records or multiple terabytes
2+ years of development experience in highly scalable, distributed systems and cluster architectures using AWS
5+ years of hands-on programming experience with Python
5+ years of professional experience working in big data ecosystems, Spark is required; PySpark is preferable
3+ years of experience with SQL, schema design, and dimensional data modeling
2+ years of professional experience working with dataflow orchestration tools, such as Airflow
2+ years of experience with non-relational databases (e.g., DynamoDB, Elasticsearch, etc.)
A bachelor’s degree in Computer Science, Information Systems, Mathematics, or a related field is required.

Benefits

100% medical/dental/vision coverage
Unlimited employee PTO
Bonus program
Equity plans
401K matching for qualified roles
Discretionary, merit-based salary increases twice a year

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data engineeringETLdata pipelinemachine learningPythonSparkPySparkSQLdataflow orchestrationnon-relational databases

Soft Skills

collaborationcommunicationproblem-solvingstakeholder engagementdata governancequality assuranceanalytical thinkingattention to detailadaptabilitytime management