Senior Data Engineer
Kpler
full-time
Posted on:
Location Type: Hybrid
Location: London • United Kingdom
Visit company websiteExplore more
Job Level
About the role
- Build and maintain Kpler's core datasets (vessels characteristics, companies, geospatial data).
- Responsible for creating and maintaining REST APIs, streaming pipelines (Kafka Stream), and Spark batch pipelines.
- Responsible for end-to-end ownership of development work, beginning with a clear understanding of assigned tickets and requirements.
- Designing and implementing functionality across APIs and data processing components.
- Ensuring deployments to development environments are tested and reviewed by peers and product stakeholders.
- Emphasize strong testing practices through unit, integration, and functional tests aligned with defined scenarios.
- Monitor performance, alerts, and SLOs to ensure the functionality operates reliably and optimally in production.
Requirements
- 6-8 years of experience in data-focused software engineering roles.
- Strong programming skills in Scala (or JVM) experience with Python preferred.
- Proven experience designing and operating RESTful APIs, including secure and versioned interfaces.
- Solid understanding of data modeling, schema evolution, versioning and serialization technologies such as Avro or Protobuf.
- Hands-on experience with SQL and NoSQL databases, including query optimization and performance tuning.
- Experience building and maintaining batch or streaming data systems, with knowledge of streaming patterns and reliability concerns.
- Familiarity with caching strategies, CI/CD pipelines, and modern monitoring and alerting practices.
- Proficiency with Git-based workflows, code reviews, and Agile development methodologies.
- Strong sense of ownership, with pragmatic problem-solving skills, constructive critique and the ability to deliver end-to-end solutions.
- Excellent communication skills and fluency in English, with the ability to collaborate effectively across product and engineering teams.
Benefits
- Professional development opportunities
- Flexible working hours
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
ScalaPythonREST APIsKafka StreamSparkSQLNoSQLAvroProtobufdata modeling
Soft skills
problem-solvingcommunicationcollaborationownershipconstructive critiquetesting practicesAgile methodologiespeer reviewpragmatismreliability