Data Engineer

• Design and build scalable data pipelines powering AI models, machine learning workflows, and intelligent product capabilities
• Own the reliability, performance, and operational excellence of production data platforms
• Develop ETL/ELT solutions using Python, SQL, AWS, and modern cloud-based data technologies
• Implement monitoring, alerting, testing, and data quality processes for trusted datasets
• Drive best practices in data governance, lineage, security, and dataset versioning
• Collaborate with cross-functional teams on AI and data solutions
• Mentor engineers and contribute to engineering standards, technical direction, and continuous improvement

Senior AI Data Engineer building AI data platforms and pipelines for Ideagen’s safety-focused software products

Senior AI Data Engineer

Senior AI Data Engineer building scalable data platforms and pipelines for Ideagen’s AI-powered safety software. Developing AWS-based ETL/ELT solutions and mentoring engineering teams.

• Design scalable data architectures and retrieval solutions across APIs, web scrapers and SFTP, with security and performance at the centre.
• Deliver prompt, efficient onboarding of CMSPI client data and contribute to new data features that create continuous business value.
• Collaborate with engineers, agile delivery leads and business stakeholders to define, refine and deliver requirements in an agile squad.
• Maintain high software quality through clean code, collaborative code and architecture reviews, automated testing and continuous integration.
• Respond to stakeholder issues and queries, troubleshoot production problems and drive practical improvements to data solutions.
• Help shape CMSPI’s next generation data platform using Databricks AI Gateway, enabling governed, agent-powered capabilities that support self-building and self-healing data pipelines.
• Participate in sprint planning and daily stand-ups, keeping stories and tasks current and raising critical blockers early.
• Learn, adopt and promote best practices in data engineering, including reusable Python libraries and maintainable functional and object-oriented code.
• Stay aware of emerging technology trends and research ideas that can improve CMSPI’s data platforms and engineering practices.
• Work collaboratively as part of an Agile development squad and contribute to a culture of continuous learning and delivery.

Data Engineer building CMSPI’s payments intelligence platform in Manchester

Data Engineer building secure ingestion, analytics workflows, and client-data solutions for CMSPI’s payments intelligence platform. Working with Azure, Databricks, Spark, and Python in a Manchester hybrid squad.

• Take operational ownership of usage and adoption pipelines and alerting
• Reconcile figures derived from the product-usage tool
• Separate development and production environments and mature CI/CD for repeatable deployment and reliable rollback
• Build and maintain Bronze-to-Gold pipelines in Databricks, Unity Catalog metric views, and permissions/row-level-security models
• Extend automated data quality checks across gold-layer tables and validate outputs against existing Power BI reports
• Co-own scoping of usage and adoption signals with Product and Post-Sales, prioritizing upsell signals and ROI narratives
• Handle schema drift, inconsistent field naming, soft deletes, and incremental loads
• Lead consolidation of additional data sources, including support tickets, audit logs, CRM, and other customer-facing systems
• Define feasibility, effort, cost, and implementation approach for a future unified data platform
• Establish documented pipelines, raw and transformed data access models, and an intake process for cross-department data requests
• Act as a point of contact for CoreView's Databricks estate
• Introduce automation and AI-assisted tooling, including monitoring and anomaly detection on metric movements

Data Engineer building governed Databricks pipelines for CoreView’s Microsoft 365 resilience platform

Data Engineer building governed Databricks pipelines and customer adoption analytics. Supporting CoreView’s Microsoft 365 tenant resilience platform with reliable data operations.

• Design and document production-grade data pipelines
• Work across ETL and ELT workflows
• Assess data movement, transformation, and processing requirements
• Produce technical deliverables based on realistic data-platform scenarios
• Develop or review SQL and Python transformations and processing logic
• Evaluate data-processing approaches for correctness and scalability
• Design or assess orchestration, dependency management, and scheduled data workflows
• Review pipeline sequencing and operational reliability considerations
• Apply streaming and large-scale distributed processing experience using technologies such as Kafka and Spark
• Evaluate distributed architecture and implementation choices
• Work with data warehouse and analytics architectures using platforms such as Snowflake, BigQuery, or Databricks
• Develop or review transformation workflows using tools such as dbt
• Assess data modelling and warehouse design decisions
• Review cloud architecture and platform decisions across AWS, GCP, or Azure
• Assess data storage, processing, and integration approaches
• Produce professional technical documentation and structured deliverables
• Explain architectural and implementation decisions clearly
• Document production practices, constraints, assumptions, and engineering considerations
• Collaborate with specialists from adjacent technical domains

UK-based Data Engineer building production pipelines and analytics infrastructure for 24-MAG consulting projects

Data Engineer – UK-Based

UK-based Data Engineer building production pipelines, orchestration, streaming, and analytics infrastructure for 24-MAG consulting projects. Producing structured technical deliverables and documentation from real-world data engineering work.

• Coordinate impact assessments, option papers and decision materials for governance forums
• Communicate decisions, conditions and implementation requirements to affected workstreams
• Control the change log, workflow, decisions, audit trail, re-baselining and implementation validation
• Work across planners, design leads, finance, risk, legal and delivery teams to secure complete assessments
• Set standards, coach contributors and drive timely closure of planning and change actions
• Challenge incomplete submissions and escalate material impacts or non-compliance with agreed decisions
• Provide senior-level planning and change control support
• Establish robust plan standards and administer end-to-end change governance
• Ensure proposed changes are fully assessed, decisions are traceable and approved impacts are reflected in the migration baseline
• Support the safe migration of TSB customers and products onto Santander platforms

Migration planning and change control for Santander’s TSB data migration programme

Migration Planning, Change Control Support – Customer & Product Data Migration Team

Migration planning and change control support for Santander’s TSB customer and product data migration. Governing impacts, decisions and baselines across regulated transformation teams.

• Manage and optimize PostgreSQL, SQL Server, and Oracle databases for performance, security, and availability
• Design and maintain database schemas, indexes, and partitions for scalability and efficiency
• Monitor databases and perform backup, recovery, and disaster recovery planning
• Implement security best practices, including access control, encryption, and auditing
• Troubleshoot database performance issues and optimize query performance
• Ensure compliance with data governance, security policies, and industry standards
• Understand business needs and translate requirements into actionable data engineering tasks
• Prioritize work and lead daily standups
• Collaborate with cross-functional teams to align Power BI and data solutions with business objectives
• Guide and support the team’s Power BI-related work without requiring direct Power BI development
• Mentor team members and drive best practices in data management and reporting
• Apply AI and automation to improve software delivery, testing, database reliability, migrations, and operational runbooks

Senior Data Engineer administering databases for M3’s global healthcare market research platforms

Senior Data Engineer, PostgreSQL DBA

Senior Data Engineer administering PostgreSQL and enterprise databases for M3’s global healthcare market research platforms. Improving performance, security, reliability, and AI-enabled data operations.

• Manage and optimize PostgreSQL, SQL Server, and Oracle databases for performance, security, and availability
• Design and maintain database schemas, indexes, and partitions for scalability and efficiency
• Perform database monitoring, backup, recovery, and disaster recovery planning
• Implement access control, encryption, auditing, and other security best practices
• Troubleshoot database performance issues and optimize query performance
• Ensure compliance with data governance, security policies, and industry standards
• Understand business needs and translate requirements into actionable data engineering tasks
• Participate in daily standups and prioritize tasks
• Lead daily standups and manage team workload alignment with business needs
• Work with cross-functional teams to support Power BI and data solutions aligned with business objectives
• Mentor team members and drive best practices in data management and reporting
• Apply AI and automation to improve software delivery, testing, reliability, and operational efficiency
• Communicate technical solutions effectively to stakeholders

Senior Data Engineer optimizing PostgreSQL for M3 Global Research’s healthcare market research platforms

Senior Data Engineer optimizing PostgreSQL databases for M3 Global Research’s healthcare market research platforms. Applying AI-driven automation, reliability, security, and performance tuning while supporting Power BI solutions.

• Own the integrated Customer & Product Migration plan
• Produce executive reporting
• Ensure design, build, remediation, testing, readiness, cutover, migration events and stabilisation are interlocked through a credible critical path, measurable milestones and controlled change
• Provide decision-led reporting on critical path, readiness, customer risk, dependencies and required interventions
• Define and control the L0-L3 plan
• Set planning standards and coordinate planners and delivery leads across horizontal and vertical workstreams
• Support governance with reliable forward-looking insight and scenario analysis
• Support the safe migration of TSB customers and products onto Santander platforms
• Help define and execute the end-to-end migration strategy

Migration planning lead coordinating TSB customer and product migrations for Santander

Migration Planning & Reporting Lead – Customer & Product Data Migration Team

Migration planning lead owning integrated plans and executive reporting. Coordinating TSB customer and product migrations onto Santander banking platforms.

• Engineer DUAL Personal Lines’ strategic data platforms, especially the Data Lakehouse
• Provide technical expertise in data engineering, analysis, orchestration, and enrichment
• Build data solutions to address changing requirements, new requests, and incidents
• Build, test, and deploy data products while maintaining governance, change control, accountability, data quality, and data security
• Consult business analysts, developers, and system owners to maintain agreed data standards
• Assess the impact of changes on the data model and prevent issues affecting operational, analytical, and reporting environments
• Collaborate with data teams and system owners to maintain the data model and data integrity across the data lakehouse
• Support underwriting analytical teams and reporting analysts with accurate data for underwriting and performance analysis and reporting
• Support delivery of API interfaces, data models, data sharing, RDM/MDM tools, and ML endpoints
• Work within Agile delivery and DevOps methodologies to deliver incremental platform changes and data products
• Work with the Group Data Science team to productionise ML pipelines
• Collaborate with stakeholders across Underwriting, Operations, Finance, Risk & Compliance, and HR
• Facilitate common approaches, standards adherence, and cooperation between DUAL Personal Lines and the DUAL Data Team
• Help drive future data investments, including articulating ROI and assessing benefits realisation

Data Engineer building Lakehouse platforms for DUAL’s international insurance underwriting business

Data Engineer building Azure Lakehouse platforms for DUAL’s international insurance underwriting business. Delivering governed data products, pipelines, models, and ML integrations for underwriting analytics.

• Contribute to the design and delivery of robust, automated, and scalable Azure and Snowflake data platform components
• Develop and maintain reusable infrastructure-as-code with Terraform across environments
• Build and optimize CI/CD pipelines using Azure DevOps and GitHub Actions
• Implement observability practices including logging, metrics, and alerting
• Collaborate with the Lead Engineer and Architects to align implementation with platform standards and patterns
• Provide technical guidance and mentorship to mid-level engineers
• Shape and operate a scalable, secure enterprise data platform supporting advanced analytics and business intelligence in healthcare and insurance
• Ensure platform compliance, performance, automation, and technical excellence

Senior Data Platform Engineer building Azure and Snowflake platforms for Bupa’s healthcare and insurance services

Senior Data Platform Engineer

Senior Data Platform Engineer building secure Azure and Snowflake platforms for Bupa’s healthcare and insurance services. Automating infrastructure, deployments, and observability.

• Engineer DUAL Personal Lines’ strategic data platforms, chiefly the Data Lakehouse, to enhance underwriting analytical capabilities
• Provide technical expertise in data engineering, analysis, orchestrations, and enrichment
• Build data solutions to address changing requirements, new requests, and incidents
• Build, test, and deploy data products in accordance with governance principles, change control, accountability, data quality, and data security
• Consult business analysts, developers, and system owners to adhere to agreed data standards
• Assess the impact of changes on the data model and prevent issues affecting operational, analytical, and reporting environments
• Collaborate with data teams and system owners to maintain the data model and data integrity across the DUAL Group data lakehouse
• Support underwriting analytical teams and reporting analysts by ensuring accurate data representation for underwriting and performance analysis and reporting
• Build knowledge of DUAL Personal Lines’ systems with other data team members
• Support delivery of project components including API interfaces, data models, data sharing, RDM/MDM tools, and ML endpoints
• Work within Agile delivery and DevOps methodologies to incrementally improve the data platform and deliver data products
• Work with the Group Data Science team to productionise ML pipelines into the data platform
• Collaborate with internal stakeholders across Underwriting, Operations, Finance, Risk & Compliance, and HR
• Act as a conduit between DUAL Personal Lines and the DUAL Data Team to share common approaches, adhere to standards, and advocate for DUAL
• Help drive future data investments, including articulating ROI and assessing realisation attainment

Data Engineer building Azure lakehouse platforms for DUAL, an international insurance underwriting group

Data Engineer building Azure lakehouse platforms for DUAL, an international insurance underwriting group. Delivering governed data products for underwriting analytics, reporting, and ML pipelines.

• Design, build and operate trusted data pipelines and data services for ESCS reporting, integration, migration and analytics
• Create scalable data pipelines and models across ESCS source and target platforms
• Build data quality, lineage, security and operational monitoring into delivery
• Support migration, reporting and decision-making with reliable data engineering
• Design and implement ingestion, transformation, validation and publishing pipelines
• Develop data models and contribute to warehouse, lakehouse or platform design
• Implement data quality controls, reconciliation, lineage and metadata
• Support master and reference data management approaches
• Engineer secure access, retention and protection controls with relevant specialists
• Plan and execute data migration, cleansing and cutover activities
• Monitor pipeline reliability, performance and cost and resolve operational issues
• Document data flows, mappings, rules, interfaces and support procedures

Data Engineer building secure data pipelines for Leidos UK’s defence and government technology solutions

Data Engineer building secure, governed data pipelines for Leidos UK’s defence, healthcare, government, and security solutions. Supporting reporting, migration, integration, and analytics.

• Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources
• Develop and optimise data architectures, including data warehouses, data lakes, and other storage solutions
• Create and maintain reliable ETL/ELT workflows to ensure data quality and accessibility
• Partner with data scientists, analysts, and client stakeholders to understand requirements and deliver impactful solutions
• Monitor and enhance data-system performance to ensure minimal downtime and rapid query responses
• Identify automation opportunities and recommend tools to improve data-engineering workflows
• Maintain detailed technical documentation for solutions and processes
• Mentor junior engineers and lead technical initiatives within cross-functional teams

Senior Data Engineer building scalable data solutions for Optima Partners’ consumer-brand consultancy

Senior Data Engineer

Senior Data Engineer building scalable pipelines and architectures for Optima Partners, an Edinburgh data and business consultancy. Delivering advanced solutions for leading consumer brands.

• Design and implement robust, scalable, and efficient data pipelines
• Collect, transform, and integrate data from various sources
• Develop and optimise data architectures, including data warehouses, data lakes, and other storage solutions
• Create and maintain reliable ETL/ELT workflows
• Work closely with data scientists, analysts, and client stakeholders to understand requirements and deliver solutions
• Monitor and enhance data-system performance to minimise downtime and improve query response times
• Identify automation opportunities and recommend tools to improve data-engineering workflows
• Maintain detailed technical documentation for solutions and processes
• Collaborate within dynamic, multi-disciplinary project teams across diverse commercial sectors

Data Engineer building scalable pipelines for Optima Partners’ consumer-brand consultancy clients

Core Data Engineer

Data Engineer building scalable pipelines and architectures for Optima Partners, a UK data and business consultancy. Delivering client-facing solutions across consumer brands and commercial sectors.

• Support the Control Tower Lead with mobilisation and execution of the Migration Control Tower
• Provide accurate integrated runbook tracking
• Collect status updates
• Control dependencies and issues
• Log decisions
• Manage communications
• Complete post-event closure
• Support the safe migration of TSB customers and products onto Santander platforms
• Contribute to the end-to-end migration strategy

Control Tower Support coordinating Santander’s TSB customer and product data migration

Control Tower Support – Customer & Product Data Migration Team

Control Tower Support coordinating Santander’s TSB customer and product data migration. Tracking runbooks, dependencies, decisions, communications, and post-event closure.

• Design, build and maintain scalable data platforms and data pipelines within AWS
• Develop ETL/ELT processes for ingesting, transforming and processing data from multiple sources
• Implement data solutions using AWS S3, Glue, Lambda, Athena, Redshift, EMR and related technologies
• Design data models and storage approaches for analytical and operational requirements
• Develop solutions using Python, SQL and modern data engineering frameworks
• Apply infrastructure-as-code and automation to deploy and manage data platforms
• Build data quality, validation, monitoring and observability into engineering solutions
• Address security, governance and data protection requirements throughout solution design and delivery
• Collaborate with architects, cloud engineers, security specialists, SMEs and client teams
• Engage clients and technical stakeholders to understand requirements, explain technical decisions and provide recommendations
• Troubleshoot complex data, performance and integration issues across development and production environments
• Define and promote data engineering standards, reusable patterns and best practices
• Mentor and support engineers and encourage knowledge sharing and continuous improvement
• Contribute to technical decisions and provide technical ownership and engineering leadership

Senior AWS Data Engineer designing secure cloud data platforms for Rayo and UBDS Group clients

Senior AWS Data Engineer

Senior AWS Data Engineer designing secure, scalable data platforms for Rayo and UBDS Group clients. Building AWS pipelines and guiding engineering practices across regulated environments.

• Translate business requirements into practical network design, reference architectures, design patterns, HLDs, design reviews, runbooks, POCs, FOAs
• Design, operate, and implement global multi-vendor, multi-cloud, hybrid data center networks at scale while driving network performance and resiliency
• Deploy and manage cloud constructs such as subnet allocations, routing-tables/associations, VPCs, VPC-peering/PrivateLink, Hub-spoke topologies, gateways, and direct connects
• Develop and maintain IaC (ansible, Terraform) and automation tooling throughout CI/CD pipeline (Python, Go) to provision networks at scale, while improving the efficiency and velocity of network deployments
• Troubleshoot complex production issues and contribute to incident root-cause analysis
• Participate in on-call rotations
• Implement network solutions meeting regional compliance and data sovereignty requirements (ISO 27001, Cyber Essentials Plus, UK G-Cloud, EU data residency / GDPR, PCI-DSS)
• Design, implement, and operate CrowdStrike's global cloud and data center network infrastructure

Senior Network Engineer designing CrowdStrike’s global cybersecurity cloud and data center infrastructure

Senior Network Engineer – Data Center, Cloud

Senior Network Engineer designing and operating CrowdStrike’s global cloud and data center networks. Building automated, resilient infrastructure for the cybersecurity platform.

• Design, build and maintain data pipelines, ETL processes and data models
• Make and document architectural and data-modelling decisions
• Integrate and transform diverse data sources into a reliable, unified data platform
• Design and implement database schemas for large volumes of structured and unstructured data
• Build scalable and maintainable data solutions supporting analytics, reporting, operational processes and data-driven products
• Champion engineering standards across SQL, Python, testing, documentation and code review
• Review designs and code produced by other engineers
• Identify and address technical debt and improve the data platform
• Optimise data solutions using partitioning, indexing and caching
• Develop and maintain data quality processes
• Improve reliability, observability and performance of data solutions
• Collaborate with product managers, developers, data colleagues and business stakeholders to define requirements
• Diagnose and resolve performance and production issues
• Coach and mentor engineers
• Keep up to date with data engineering developments and evaluate new technologies
• Participate in the team’s out-of-hours support rota where required

Senior Data Engineer building Travel Chapter’s holiday-rental data platform

Senior Data Engineer building Travel Chapter’s data platform for holiday experiences. Designing pipelines, warehouses and reliable datasets for analytics, operations and data products.

• Build and maintain ingestion pipelines into the data warehouse using Fivetran and custom pipelines
• Own data orchestration to keep data moving on schedule and recover cleanly from failures
• Build and manage AWS-based data infrastructure as code with Terraform
• Build enterprise-grade customer data pipelines and connectors for HR, payroll, and other systems
• Reliably and securely move customer data in and out at enterprise volumes
• Connect data to AI agents and services, including through MCP connectors
• Work with product and stakeholders to scope problems and agree on simple, scalable solutions
• Build according to security practices such as the OWASP Top 10 and AWS Well-Architected Framework

Software Engineer building reliable data infrastructure for Ben’s global employee benefits platform

Software Engineer, Data

Software Engineer building Python, SQL, AWS, and Terraform data infrastructure for Ben’s global employee benefits platform. Operating enterprise pipelines, integrations, and AI-agent data services.

• Design, develop, and maintain scalable data pipelines, integrations, and transformation processes across multiple data sources
• Lead the implementation and optimisation of Microsoft Fabric and associated cloud-based data engineering solutions
• Build and maintain enterprise data products, data models, and structured datasets supporting reporting, analytics, and operational decision-making
• Establish and apply data quality, validation, and assurance frameworks
• Collaborate with project teams, business stakeholders, analysts, and digital specialists to define and deliver data requirements
• Drive continuous improvement through automation, process optimisation, and industry best practices
• Ensure compliance with standards relating to information management, data governance, cyber security, and data privacy
• Troubleshoot and resolve complex data-related issues, providing technical expertise and guidance
• Produce and maintain technical documentation, architecture diagrams, standards, and operational procedures
• Mentor junior team members through knowledge sharing and technical leadership
• Contribute to organisational data engineering standards, frameworks, and reusable solutions
• Support solution architecture, platform design, and strategic data initiatives across programmes and business functions

Senior Data Engineer building enterprise data solutions for Arcadis’ sustainable design and engineering projects

Senior Data Engineer building Microsoft Fabric and Azure data solutions for Arcadis’ sustainable design, engineering, and consultancy projects. Enabling secure analytics, reporting, and data-driven decisions.

Data Engineer jobs in United Kingdom

See all jobs on JobTailor

Senior AI Data Engineer

Data Engineer

Data Engineer

Data Engineer – UK-Based

Migration Planning, Change Control Support – Customer & Product Data Migration Team

Senior Data Engineer, PostgreSQL DBA

Senior Data Engineer, PostgreSQL DBA

Migration Planning & Reporting Lead – Customer & Product Data Migration Team

Data Engineer

Senior Data Platform Engineer

Data Engineer

Data Engineer

Senior Data Engineer

Core Data Engineer

Control Tower Support – Customer & Product Data Migration Team

Senior AWS Data Engineer

Senior Network Engineer – Data Center, Cloud

Senior Data Engineer

Software Engineer, Data

Senior Data Engineer