Design, build, and maintain batch data pipelines using Databricks and Azure cloud services.
Develop and curate analytics-ready datasets using modern lakehouse patterns.
Support ad-hoc data integration and extraction needs for the Advanced Analytics team.
Serve as the primary Databricks practitioner for the Advanced Analytics team, using Spark, SQL, and Python.
Collaborate with central Technology and Data Architecture teams to align with platform standards and best practices.
Identify and recommend improvements to tools, workflows, and data engineering patterns.
Implement data quality checks and basic monitoring to ensure reliable analytics datasets.
Use Git-based workflows and support CI/CD practices across development, test, and production environments.
Troubleshoot and resolve pipeline issues in a timely manner.
Partner with Data Scientists to translate analytical needs into scalable data solutions.
Enable self-serve analytics by delivering well-structured, documented datasets.
Support rapid experimentation, pilots, and ongoing analytical use cases.
Provide mentorship, task guidance, and code reviews for a Junior Data Engineer.
Contribute to team planning, documentation, and continuous process improvement initiatives.

Requirements

Bachelor’s degree in Computer Science, Engineering, or a related technical field.
One or more relevant cloud or data engineering certifications (e.g., Azure, Databricks, or equivalent).
3+ years of experience designing, building, and maintaining data pipelines or data platforms in a production environment.
Hands-on experience with Databricks, Apache Spark, or equivalent distributed data processing frameworks.
Demonstrated experience mentoring or coaching team members particularly in technical or leadership development contexts.
Experience working in a cloud environment, preferably Microsoft Azure.
Experience with Delta Lake and lakehouse architectural patterns is preferred.
Experience supporting analytics or data science teams is preferred.
Strong programming experience in Python and SQL; experience with Spark SQL preferred.
Familiarity with CI/CD practices for data engineering workloads.
Solid understanding of modern data warehousing and lakehouse concepts.
Exposure to data governance, access control, and enterprise data is preferred.

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

data pipelinesDatabricksAzureSparkSQLPythonDelta LakeCI/CDdata quality checksdata engineering

mentorshiptask guidancecollaborationtroubleshootingprocess improvementcommunicationleadership development

Bachelor’s degree in Computer ScienceBachelor’s degree in EngineeringAzure certificationDatabricks certification