Tech Stack
AzureCloudDockerKubernetesMongoDBPythonSparkTerraformVault
About the role
- Design, implement, and maintain data infrastructure leveraging Azure, Databricks, and MongoDB.
- Manage and optimize Azure resources (ADLS, Blob Storage, Azure Kubernetes Service, networking, Key Vault, Private Endpoints, Vnets, etc.) for scalable and secure data operations.
- Deploy and maintain Databricks clusters, jobs, and pipelines with a focus on performance tuning and cost optimization.
- Operate and scale MongoDB (on Azure or Atlas), including schema design, indexing, sharding, and replication strategies.
- Build and automate infrastructure-as-code (IaC) solutions using tools such as Terraform, Bicep, or ARM templates.
- Support a managed Snowflake environment and associated transformation tooling like dbt and integrations with AzureDevOps.
- Ensure compliance with security, governance, and data residency requirements in a cloud environment.
- Implement monitoring, alerting, and observability solutions for proactive infrastructure management.
- Support disaster recovery, backup, and incident response processes.
- Maintain documentation about the deployed infrastructure and coordinate project deployments to align with the overall architecture and current and future requirements.
- Collaborate with other developers, architects, development teams and stakeholders.
Requirements
- Master's degree in computer science, computer engineering, management information systems, related discipline or equivalent experience in data infrastructure or cloud engineering
- Strong expertise with Azure cloud services, including networking, storage, and security
- Hands-on experience with Databricks (cluster management, Delta Lake, Spark optimization)
- Proficiency in MongoDB administration and scaling (performance tuning, high availability, ckups)
- Experience with IaC tools (Terraform, Bicep, ARM) for reproducible infrastructure deployments
- Proficiency in scripting and automation (Python, PowerShell, Bash)
- Experience with CI/CD practices for infrastructure and data pipelines
- Experience with Azure DevOps or GitHub Actions for deployment automation
- Familiarity with container orchestration (AKS, Kubernetes, Docker)
- Experience supporting Snowflake environments and transformation tooling like dbt
- Ensure compliance with security, governance, and data residency requirements in a cloud environment
- Experience implementing monitoring, alerting, and observability solutions
- Experience with disaster recovery, backup, and incident response processes
- Strong collaboration skills; ability to work with developers, architects, development teams and other stakeholders