Lead Data Engineer, Databricks – Manufacturing/MES
CompuGain
contract
Posted on:
Location Type: Remote
Location: California • United States
Visit company websiteExplore more
Job Level
About the role
- Design, build, and optimize scalable data pipelines using Azure Databricks (PySpark, Spark SQL, Delta Lake)
- Integrate and process MES data, shop-floor data, IoT/sensor data, and ERP systems
- Architect modern data solutions supporting manufacturing analytics and operational reporting
- Lead data engineering best practices including performance tuning, optimization, and governance
- Work closely with manufacturing stakeholders to translate MES data requirements into scalable data models
- Ensure high data quality, reliability, and production-grade deployments.
Requirements
- Strong hands-on expertise in Azure Databricks (Delta Lake, PySpark, Spark architecture)
- Proven experience in Manufacturing domain, especially working with MES systems and production data
- Experience building real-time or near-real-time data pipelines
- Strong SQL and data modeling skills
- Experience with cloud platforms (Azure preferred)
- Ability to function as both technical architect and hands-on developer
- Excellent communication skills and ability to work in PST time zone
- Preferred: Experience with IoT data ingestion and streaming technologies
- Exposure to data warehousing and BI integration
- Experience leading small data engineering team
Benefits
- All your information will be kept confidential according to EEO guidelines.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
Azure DatabricksPySparkSpark SQLDelta LakeSQLdata modelingreal-time data pipelinesdata warehousingBI integrationIoT data ingestion
Soft Skills
communicationleadershipcollaborationproblem-solvingperformance tuningoptimizationgovernancestakeholder engagementteam leadershipadaptability