UBDS

Data Engineer – AWS

UBDS

full-time

Posted on:

Location Type: Hybrid

Location: London • United Kingdom

Visit company website

Explore more

✨ AI Apply
Apply

Tech Stack

AWSAzureDistributed SystemsKafkaPySparkPythonTerraform

About the role

  • Design, develop, and maintain data pipelines and transformation processes using modern data engineering tools
  • Build and manage batch and real-time data processing solutions
  • Develop curated, reusable data assets to support downstream analytics and reporting
  • Ensure data quality, integrity, and consistency across multiple data sources
  • Work with data ingested via APIs, file-based ingestion (e.g. SFTP), and event streams (e.g. Kafka)
  • Optimise data pipelines for performance, scalability, and cost efficiency
  • Contribute to the development of data platform components and reusable engineering patterns
  • Work within established architecture and engineering standards
  • Collaborate with DevOps engineers, testers, and architects to deliver end-to-end data solutions
  • Support onboarding of new data sources and integration into existing platforms
  • Contribute to knowledge sharing and continuous improvement within engineering teams

Requirements

  • Experience in data engineering, building and maintaining data pipelines
  • Strong hands-on experience with Python and PySpark
  • Experience working with AWS data services (e.g. S3, EMR, Glue)
  • Experience with event-driven architectures, particularly Kafka
  • Understanding of batch and real-time data processing
  • Experience working with large-scale datasets and distributed systems
  • Familiarity with data quality, validation, and schema management
  • Experience using Terraform and GitHub within a DevOps environment
  • Ability to work within defined engineering standards and best practices
  • Experience with Azure, Databricks, or Microsoft Fabric
  • Experience in regulated environments
  • Exposure to data migration from legacy systems
  • Understanding of analytics, reporting, or data product use cases
  • Experience working in collaborative, cross-functional teams
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
data engineeringdata pipelinesPythonPySparkAWS data servicesKafkaTerraformGitHubAzureDatabricks
Soft Skills
collaborationknowledge sharingcontinuous improvementproblem-solvingcommunicationteamworkadaptabilityattention to detailorganizational skillsability to work within standards