Design, build, and maintain batch data pipelines using Databricks and Azure cloud services
Develop and curate analytics-ready datasets using modern lakehouse patterns (e.g., Delta Lake)
Support ad-hoc data integration and extraction needs for the Advanced Analytics team
Serve as the primary Databricks practitioner for the Advanced Analytics team, using Spark, SQL, and Python
Collaborate with central Technology and Data Architecture teams to align with platform standards and best practices
Identify and recommend improvements to tools, workflows, and data engineering patterns
Implement data quality checks and basic monitoring to ensure reliable analytics datasets
Use Git-based workflows and support CI/CD practices across development, test, and production environments
Troubleshoot and resolve pipeline issues in a timely manner
Partner with Data Scientists to translate analytical needs into scalable data solutions
Enable self-serve analytics by delivering well-structured, documented datasets
Support rapid experimentation, pilots, and ongoing analytical use cases
Provide mentorship, task guidance, and code reviews for a Junior Data Engineer
Contribute to team planning, documentation, and continuous process improvement initiatives

Requirements

Bachelor’s degree in Computer Science, Engineering, or a related technical field
One or more relevant cloud or data engineering certifications (e.g., Azure, Databricks, or equivalent) are required
3+ years of experience designing, building, and maintaining data pipelines or data platforms in a production environment
Hands-on experience with Databricks, Apache Spark, or equivalent distributed data processing frameworks
Demonstrated experience mentoring or coaching team members particularly in technical or leadership development contexts
Experience working in a cloud environment, preferably Microsoft Azure (e.g., Azure Data Lake, Blob Storage, Azure Data Factory)
Experience with Delta Lake and lakehouse architectural patterns is preferred
Experience supporting analytics or data science teams is preferred
Strong programming experience in Python and SQL; experience with Spark SQL preferred
Familiarity with CI/CD practices for data engineering workloads
Solid understanding of modern data warehousing and lakehouse concepts
Exposure to data governance, access control, and enterprise data is preferred.

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

data pipelinesDatabricksAzureDelta LakeSparkSQLPythonCI/CDdata quality checksdata engineering patterns

mentorshiptask guidancecode reviewscollaborationproblem-solvingcommunicationcontinuous process improvement

Azure certificationDatabricks certification