dsm-firmenich

Associate Manager, D&T, Data Engineering

dsm-firmenich

full-time

Posted on:

Location Type: Office

Location: Hyderabad • India

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

AWSCloudPySparkPythonSparkSQLTerraformUnityVault

About the role

  • Design and maintain scalable batch and real-time data pipelines on Databricks using PySpark, Structured Streaming, DBT, and Delta Live Tables.
  • Manage cloud infrastructure with Terraform on AWS and deploy workflows using Databricks Asset Bundles with CI/CD best practices.
  • Optimize Spark performance and costs while implementing data governance through Unity Catalog.
  • Collaborate with Data Scientists to produce ML models using MLflow and ensure data quality through automated testing and robust deployment pipelines.
  • Develop and manage robust transformation layers using DBT to deliver clean, tested, and well-documented data models.
  • Enforce data governance and quality standards using Unity Catalog (row-level security) and support MLOps initiatives through MLflow-based model tracking and production deployment.

Requirements

  • 10+ years of experience in Data Engineering, with strong expertise in the Databricks platform.
  • Deep hands-on experience with Unity Catalog, Delta Live Tables (DLT), and Databricks Asset Bundles (DAB).
  • Strong proficiency in AWS services (S3, Lambda, Kinesis) and Infrastructure as Code using Terraform.
  • Expert-level programming skills in Python (PySpark) and SQL for scalable data processing.
  • Proven experience in data transformation and modeling using DBT.
  • Solid understanding of data modeling concepts, including Star Schema, Dimensional Modeling, and Data Vault, with the ability to contribute to architectural design decisions.
Benefits
  • A team of diverse employees who aren’t afraid to think outside of the box.
  • A truly global and collaborative team that cares about the experience of our employees.
  • The encouragement you need to develop and achieve personal growth.
  • A role that is crucial on projects and allows you to build your brand.
  • A caring and supportive environment where you’re empowered to grow and share your ideas.
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
PySparkStructured StreamingDBTDelta Live TablesTerraformAWSSQLdata transformationdata modelingMLOps