GCP Data Engineer
KI
full-time
Posted on:
Location Type: Hybrid
Location: City of London • United Kingdom
Visit company websiteExplore more
About the role
- Work with both the business teams (finance and actuary initially), data scientists and engineers to design, build, optimise and maintain production grade data pipelines and reporting from an internal Data warehouse solution, based on GCP/Big Query
- Work with finance, actuaries, data scientists and engineers to understand how we can make best use of new internal and external data sources
- Work with our delivery partners at EY/IBM to ensure robustness of Design and engineering of the data model/ MI and reporting which can support our ambitions for growth and scale
- BAU ownership of data models, reporting and integrations/pipelines
- Create frameworks, infrastructure and systems to manage and govern Ki’s data asset
- Produce detailed documentation to allow ongoing BAU support and maintenance of data structures, schema, reporting etc.
- Work with the broader Engineering community to develop our data and MLOps capability infrastructure
- Ensure data quality, governance, and compliance with internal and external standards.
- Monitor and troubleshoot data pipeline issues, ensuring reliability and accuracy.
Requirements
- Experience designing data models and developing industrialised data pipelines
- Strong knowledge of database and data lake systems
- Hands on experience in Big Query, dbt, GCP cloud storage
- Proficient in Python, SQL and Terraform
- Knowledge of Cloud SQL, Airbyte, Dagster
- Comfortable with shell scripting with Bash or similar
- Experience provisioning new infrastructure in a leading cloud provider, preferably GCP
- Proficient with Tableau Cloud for data visualization and reporting
- Experience creating DataOps pipelines
- Comfortable working in an Agile environment, actively participating in approaches such as Scrum or Kanban
- Desirable Skills
- Experience of streaming data systems and frameworks would be a plus
- Experience working in regulated industry, especially financial services would be a plus
- Experience creating MLOps pipelines is a plus
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data modelingdata pipelinesBig QuerydbtPythonSQLTerraformCloud SQLAirbyteMLOps
Soft Skills
collaborationcommunicationproblem-solvingdocumentationagile methodologiesparticipation in Scrumparticipation in Kanbandata governancedata quality assurancetroubleshooting