Senior Data Engineer

Quantum

Senior Data Engineer helping to design and build scalable real-time data pipelines for industry-leading digital comparison service. Collaborating across teams for high performance data processing.

Posted 4/12/2026full-timeLondon • 🇬🇧 United KingdomSeniorWebsite

Tech Stack

Tools & technologies

AirflowApacheAWSAzureCloudGoogle Cloud PlatformJavaKafkaKubernetesNoSQLPythonScalaSparkSQL

About the role

Key responsibilities & impact

Shape, develop, and maintain scalable real-time data pipelines and infrastructure
Integrate various data sources and ensure seamless real-time data flow across the organisation
Build efficient, fault-tolerant, and highly available data ingestion processes
Monitor and improve data pipeline performance and scalability for low-latency and high-throughput
Partner closely with cross-functional teams - including software engineers, data scientists, and business stakeholders - to ensure data infrastructure aligns with business objectives

Requirements

What you’ll need

Solid hands-on experience in data engineering, with a strong track record of building and managing real-time data pipelines across multiple initiatives
Strong expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.)
Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage
Programming skills in Python, Java, Scala, or a similar language
Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling
Solid understanding of data pipeline orchestration tools (e.g. Apache Airflow, Kubernetes)
Excellent written and verbal English skills.

Benefits

Comp & perks

Private Health Care (Bupa)
Hybrid Working (3 days in office)
Travel Insurance
Competitive Base Salary
Company Bonus Scheme
Market-Leading Training Programme
Recognition & Reward Scheme
Annual Company Conference (previous destinations: Vienna, Bologna, Dubrovnik, Thessaloniki)
Regular Happy Hours & Team Lunches
Free Coffee, Drinks & Snacks

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data engineeringreal-time data pipelinesdistributed streaming platformsKafkaSpark StreamingFlinkcloud platformsAWSGCPAzure

Soft Skills

collaborationcommunicationproblem-solvingcross-functional teamworkwritten communicationverbal communication