EXL

Data Engineer, Databricks

EXL

full-time

Posted on:

Location Type: Hybrid

Location: London • 🇬🇧 United Kingdom

Visit company website
✨ AI Apply
Apply

Job Level

SeniorLead

Tech Stack

AWSAzureCloudETLGoogle Cloud PlatformPySparkSparkSQLVault

About the role

  • Design, build, and deploy scalable data pipelines using Azure Databricks, Azure Data Factory, and Azure SQL Database
  • Optimise Databricks environments for performance, cost, and security
  • Develop and maintain data integration, ETL/ELT processes, and data quality frameworks
  • Translate business requirements into robust data models, ensuring consistency and accuracy across data layers
  • Work closely with stakeholders to deliver effective data and analytics solutions
  • Produce technical documentation, including data flow diagrams, architecture designs, and test plans
  • Stay current with emerging data engineering trends and Azure ecosystem technologies to drive continuous improvement

Requirements

  • 8–10 years’ experience in data engineering and integration, ideally in enterprise-scale environments
  • Minimum of 5 years’ management or technical leadership experience, leading delivery or overseeing project teams
  • Deep expertise in Azure Databricks, Azure Data Factory, Azure Synapse, and Azure Data Lake
  • Strong hands-on experience with Spark (PySpark / SparkSQL) and data pipeline orchestration
  • Solid data modelling skills (Dimensional, ODS, Data Vault) and data warehousing concepts
  • Experience working with large, complex datasets in distributed computing environments
  • Proficiency in SQL, version control tools (e.g. Git), and CI/CD pipelines
  • Experience working in Agile and DevOps environments (Azure DevOps, JIRA, etc.)
  • Advantageous Skills & Experience: Experience working within the Insurance domain, including exposure to policy, claims, or regulatory data
  • Knowledge of Enterprise Data Management (EDM) practices, data governance, metadata management, and lineage tools
  • Exposure to multi-cloud environments (AWS or GCP)
  • Understanding of modern data architecture approaches, including Dimensional, ODS, and Data Vault, as well as real-time cloud data capabilities
Benefits
  • A competitive salary with a generous bonus
  • Private healthcare
  • Critical illness life assurance at 4 x your annual salary
  • Income protection insurance
  • A rewarding pension
  • Everyday financial well-being solutions, such as cash back cards
  • Cycle Scheme where you can save money on bikes and cycling accessories
  • Wide range of professional and personal development opportunities
  • Support for learning initiatives, online courses, seminars, and workshops
  • Eligible to purchase stock as part of Employee Stock Purchase Plan (ESPP)
  • Flexible hybrid working model

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills
data engineeringdata integrationETLELTdata quality frameworksdata modellingSQLSparkdata warehousingcloud data capabilities
Soft skills
leadershipstakeholder engagementtechnical documentationcommunicationcollaborationproblem-solvingadaptabilitycontinuous improvement