Data Engineer III

Dyson

Data Engineer III designing and building complex data pipelines for AI/ML capabilities in the leading firm Robert Half. Collaborating with cross-functional teams to optimize data infrastructure and ensure data quality.

Posted 5/24/2026full-timeRemote • California • 🇺🇸 United StatesMid-LevelSenior💰 $104,000 - $153,000 per yearWebsite

Tech Stack

Tools & technologies

AWSCloudDockerETLHerokuKubernetesPythonSDLCSparkSQLTerraformUnity

About the role

Key responsibilities & impact

Lead architecture and design of complex data pipelines on Databricks lakehouse architecture (Unity Catalog, Delta Lake, Structured Streaming)
Define technical approach for data engineering initiatives, mentor less-senior engineers, and set standards for code quality through leadership and code reviews
Design and build data foundations that enable AI/ML capabilities — feature stores, embedding pipelines, vector search indexes, and model training datasets
Align data engineering solutions with business strategy, including support for Agentic AI workloads
Own health, scalability, and modernization of data infrastructure with Databricks as the strategic platform — including workload migration, compute optimization, and Unity Catalog adoption
Optimize pipeline performance (Delta Lake table layouts, clustering, Z-ordering) and establish monitoring/alerting best practices with clear SLAs
Build data infrastructure supporting Agentic AI systems — real-time data access layers, context retrieval pipelines, and agent-accessible data services
Collaborate cross-functionally with DevOps, Platform Engineering, and MLOps roles to integrate data solutions into the broader technology environment and shared AI infrastructure – Mlflow registries, feature stores, and agent orchestration layers
Provide consultation to Senior Leadership on complex projects and drive continuous improvement initiatives
Champion data governance at all layers for data, models, and AI assets
Implement data quality strategies (master data management, validation rules, Delta Live Tables expectations) to ensure trust in enterprise data
Serve as liaison across data engineering, AI engineering, and business teams; promote data literacy and stewardship

Requirements

What you’ll need

Bachelor's in Computer Science, Engineering, or related field (Master's preferred)
5+ years with Python and SQL in data engineering for big data ML/analytics workloads
5+ years designing, building, and troubleshooting scalable ETL/ELT pipelines for business-critical production systems
3+ years with cloud data services (AWS), container orchestration (Docker, Kubernetes), and IaC (Terraform, CloudFormation)
3+ years architecting ML workflows and data platforms with CI/CD, automated testing, and distributed processing (Spark)
3+ years collaborating cross-functionally with Data Science, MLOps, Platform Engineering, and DevOps teams
3+ years implementing data quality testing and optimizing SQL/Python for cost/performance in the cloud
Understanding of the full Data Science SDLC, and experience mentoring engineers
Strongly Preferred - Databricks & AI Platform
2+ years hands-on with Databricks (Delta Lake, Unity Catalog, Databricks SQL)
Experience with MLflow experiment tracking and model registry workflows
Experience designing pipelines that serve AI/ML inference — real-time feature engineering, embedding generation, and context retrieval for LLM-based systems
Understanding of how data engineering supports Agentic AI: agent-accessible data services, low-latency retrieval, and pipelines enabling autonomous multi-step workflows
Familiarity with Databricks Mosaic AI, Vector Search, and/or Feature Store
FinOps awareness — compute cluster optimization, cost attribution by workload
Familiarity with Salesforce/Heroku data infrastructures
Experience with data virtualization (e.g., Dremio)
Understanding of Platform Engineering concepts and internal developer platforms
Experience migrating from legacy data warehouse/lake to unified lakehouse architecture
Familiarity with Odaseva data security and management

Benefits

Comp & perks

group health insurance benefits (medical, vision, dental)
FSA and HSA healthcare accounts
life and accident insurance
adoption and fertility assistance
paid parental leave of up to 6 weeks
short/long term disability
paid time off for vacation, personal needs, and sick time
up to 17 days of Choice Time Off (CTO) per calendar year
up to 11 paid holidays per calendar year
opportunity to contribute to company's 401(k) savings and investment plan or deferred compensation plan with an employer match of 100% on the first 3% of contributions

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

PythonSQLETLELTDatabricksDelta LakeUnity CatalogML workflowsCI/CDSpark

Soft Skills

leadershipmentoringcollaborationconsultationdata literacycontinuous improvementdata governancecommunicationproblem-solvingcross-functional teamwork

Certifications

Bachelor's in Computer ScienceMaster's in related field