Senior AI Data Pipeline Engineer – Autonomous Driving

42dot

AI Data Pipeline Engineer developing reliable data extraction and labeling pipelines for autonomous driving technologies. Building infrastructure to support ML model training and deployment in cloud environments.

Posted 4/29/2026full-timeSunnyvale • California • 🇺🇸 United StatesSenior💰 $133,000 - $254,000 per yearWebsite

Tech Stack

Tools & technologies

AirflowApacheBootstrapCloudMongoDBPostgresPythonSpark

About the role

Key responsibilities & impact

Develop high scale, reliable data extraction pipeline to extract millions of raw data from data collection fleet and convert to high-value scene data
Develop data labeling pipelines to perform the auto labeling inferences for autonomous driving algorithms
Develop advanced autonomous driving data SDK, including scene data search, datasets preparation, dataset loading, etc.
Build up the data lakehouse for autonomous driving scene dataset, including the sensor data, calibration data, as well as annotation data
Dig into performance bottlenecks all along the data processing pipelines, from data processing latency, data search latency to Test Procedure (TP) coverage.
Bootstrap and maintain infrastructure for data platform components—data processing pipeline, database, data lakehouse and data serving.
Collaborate with cross-functional teams, including ML algorithm, ML application, and Cloud Infra to align data pipelines with overall autonomous driving system architecture.

Requirements

What you’ll need

Bachelor's degree or higher in Computer Science, Engineering, Robotics, or a similar technical field.
Minimum of 7 years of experience in Data Engineering, DataOps or ML Platform roles
Proficient in Python and solid experience in Python SDK development
Solid hands-on experience with data pipeline job orchestration with Databricks Workflows or Apache Airflow, as well as integrating data pipelines with machine learning models
Solid working experience in Databases (e.g., MongoDB, PostgreSQL, etc)
Extensive experience with data technologies and architectures such as Data Warehouse (e.g., Hive) or Lakehouse (e.g., Delta Lake)
Experience with Apache Spark or other big data computing engines
Excellent leadership and communication skills, with a demonstrated ability to lead technical projects.

Benefits

Comp & perks

Health insurance
401(k)
Professional development opportunities
Flexible work arrangements
Bonuses

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Pythondata pipeline orchestrationDatabricks WorkflowsApache AirflowMongoDBPostgreSQLData WarehouseLakehouseApache Sparkdata extraction

Soft Skills

leadershipcommunication