Lead the design and development of complex data pipelines for data collection and processing.
Develop and maintain advanced data storage solutions.
Ensure data quality and consistency through sophisticated validation and cleansing processes.
Collaborate with cross-functional teams to understand data requirements and provide innovative solutions.
Optimize data engineering processes for performance, scalability, and reliability.

Requirements

3+ years relevant experience and a Bachelor’s degree OR Any equivalent combination of education and experience
Expert-level SQL skills - ability to write and optimize complex queries across large datasets (billions of records)
Strong experience with both batch processing (Spark, Dataproc) and real-time/streaming systems (Kafka, Pub/Sub, Dataflow)
Proven ability to build low-latency data systems that solve time-sensitive business problems
Hands-on proficiency with Cloud data services. Preferably - BigQuery, Bigtable, Dataproc, Cloud Storage, Pub/Sub
Strong Python programming skills for data pipeline development and automation
Experience with workflow orchestration tools (Airflow, UC4, or similar)
Solid understanding of data modeling, schema design, and data warehouse concepts
Experience with CI/CD pipelines and DevOps practices for data systems
Strong debugging and troubleshooting skills for distributed data systems
Ability to work in a fast-paced environment and ramp up quickly on new technologies and domains

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

SQLPythonSparkDataprocKafkaPub/SubDataflowdata modelingschema designCI/CD

Soft Skills

collaborationproblem-solvingdebuggingtroubleshootingadaptability