Databricks Platform Engineer

EXL

Databricks Platform Engineer managing AWS integration and data pipelines for advanced analytics. Driving innovation and optimization in Databricks ecosystem with a focus on machine learning capabilities.

Posted 4/11/2026full-time🇨🇴 ColombiaMid-LevelSeniorWebsite

Tech Stack

Tools & technologies

AWSCloudPySparkPythonTerraformUnity

About the role

Key responsibilities & impact

Leads the design, build, and optimization of the Databricks ecosystem, including Fivetran ingestion and dbt transformation pipelines.
Ensures scalable, reliable compute environments, seamless AWS integration, and automated deployments using Terraform and GitHub Actions.
Manage Fivetran connectors and dbt projects for end-to-end data ingestion, transformation, and orchestration across Databricks, S3, and Snowflake.
Automate infrastructure provisioning and configuration using Terraform integrated with GitHub Actions.
Architect and implement high availability (HA) and disaster recovery (DR) strategies, including cross-region replication and failover.
Implement job orchestration, cluster policies, autoscaling, and CI/CD workflows for notebooks, models, and releases.
Establish monitoring and alerting for cost, usage, pipeline health, data freshness, and compliance using tools such as Datadog.
Collaborate with platform segments and security teams to enforce governance, tagging, rightsizing, SLAs, and audit readiness.
Develop and maintain Unity Catalog for governance, lineage, and secure data sharing across platforms.
Build and maintain MLflow integrations for model lifecycle management, experimentation tracking, and deployment.

Requirements

What you’ll need

Graduate in Computer Science, Data Science, or related field.
4-6 years of experience in data engineering or related field.
Deep expertise in Databricks administration, cluster management, job scheduling, Unity Catalog, Fivetran, and dbt (Core/Cloud).
Proficiency in Terraform, GitHub Actions, and Python/PySpark for automation.
Strong knowledge of AWS services, including EKS, ECS, IAM, S3, VPC networking, and CloudWatch.
Experience in pipeline reliability engineering, cost optimization, ML CI/CD, HA/DR design, and observability.

Benefits

Comp & perks

Health insurance
401(k) matching
Flexible work hours
Paid time off
Professional development opportunities

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

DatabricksFivetrandbtTerraformGitHub ActionsPythonPySparkAWSMLflowUnity Catalog