Develop features for the schedule-based processing framework built on top of Airflow, AWS EMR, and DuckDB.
Improve the stability, performance, and scalability of our data ingestion and processing platforms as we scale our geofence and robotaxi deployment.
Collaborate with cross-functional teams, such as software engineers, data scientists, data engineers, and TPMs, to gather requirements, design robust architectures, and implement effective solutions.
Partner with Staff and Senior engineers inside and outside the organization to translate user pain points into concrete technical solutions and roadmap items.
Enhance system observability by building monitoring and alerting tools to track performance and measure success.

Requirements

Bachelor's or Master's in Computer Science or related fields with 4+ years of industry experience in software engineering
Strong background in Python for large-scale data processing
Familiarity with large-scale data processing systems like Spark, Trino, and DuckDB
Experience in using cloud services, such as AWS, GCP, or Azure
Strong experience in troubleshooting data pipeline failures and optimizing pipeline performance and cost efficiency

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

PythonAirflowAWS EMRDuckDBSparkTrinodata processingdata pipeline optimizationtroubleshootingscalability

Soft Skills

collaborationcommunicationproblem-solvingrequirements gatheringarchitecture designtechnical solution developmentmonitoringalerting

Certifications

Bachelor's in Computer ScienceMaster's in Computer Science