Design and manage scalable data ingestion pipelines using Azure Data Factory and Azure Service Bus for batch and near real-time processing
Implement Bronze, Silver, and Gold data layer transformations using Databricks (PySpark, SparkSQL) within a lakehouse architecture
Integrate structured, semi-structured, and unstructured data sources using formats such as Parquet, Avro, and JSON into ADLS Gen2
Establish data quality frameworks and validation processes using Databricks and Azure Data Factory
Configure RBAC and ACL-based access controls to secure sensitive datasets and ensure compliance
Manage credential security with Azure Key Vault and enforce governance standards across data platforms
Automate pipeline orchestration using Databricks Workflows and CI/CD tools such as Azure DevOps, GitHub Actions, and Terraform
Monitor pipeline health and performance using Azure Monitor, logging, and alerting to meet SLA requirements
Optimize ADLS storage performance and manage cloud costs using Azure Cost Management best practices

Requirements

Minimum 7–10 years experience
Bachelor’s degree in Computer Science, Information Systems, Data Science, or a related field (Master’s preferred)
Azure Data Factory for ETL/ELT operations
Databricks (PySpark, SparkSQL) for data transformations
SQL Server, PostgreSQL, Cosmos DB, CRM, and ERP systems
Data Governance, RBAC, and ACLs for managing permissions
CI/CD tools: Azure DevOps, GitHub Actions, Terraform
Azure Monitoring, Alerting, and Logging tools

Benefits

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Azure Data FactoryDatabricksPySparkSparkSQLSQL ServerPostgreSQLCosmos DBETLELTData Governance