Principal Data Engineer

Compare the Market

Principal Data Engineer designing data platform architecture and strategy for Compare the Market. Leading engineering best practices and ensuring scalability, resilience, and cost efficiency.

Posted 4/17/2026full-timeLondon • 🇬🇧 United KingdomLeadWebsite

Tech Stack

Tools & technologies

AWSCloud

About the role

Key responsibilities & impact

Own the Data Platform architecture and strategy, defining the long-term target state and setting clear standards and guardrails to ensure scalability, resilience, and cost efficiency.
Set the engineering bar by establishing best practices across data ingestion, streaming, transformation, modelling, CI/CD, orchestration, testing, observability, and governance.
Act as the senior technical authority across data engineering, influencing architecture decisions and leading the Data Engineering Community of Practice to raise capability across the organisation.
Champion “data as a product”, ensuring datasets and platform services are reliable, discoverable, reusable, and governed with clear ownership and service expectations.
Drive strategic technical direction while remaining hands-on where it matters—reviewing designs, validating patterns, and guiding complex problem-solving without becoming a delivery bottleneck.

Requirements

What you’ll need

Extensive experience in Data Engineering within modern, cloud-based data platforms.
Deep expertise in streaming-first architectures, with strong knowledge of batch processing approaches.
Strong experience across AWS data ecosystems, Databricks, dbt, CI/CD practices, and orchestration frameworks.
Proven track record of defining and implementing large-scale platform architecture and engineering standards.
Strong understanding of data modelling, ingestion patterns, performance optimisation, observability, governance, and security.
Experience influencing senior engineers and stakeholders across multiple teams without formal authority.
Experience leading engineering communities of practice and raising technical standards at organisational scale.
Ability to operate independently in complex, high-impact environments.

Benefits

Comp & perks

Flexible working hours
Professional development opportunities

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data engineeringcloud-based data platformsstreaming architecturesbatch processingdata modellingdata ingestionperformance optimisationobservabilitygovernancesecurity

Soft Skills

influencingleadershipproblem-solvingindependencecommunication