Data Engineer

• Build, maintain, and optimise scalable data ingestion and analysis pipelines using workflow engines such as Nextflow.
• Translate scientific and analytical prototypes into robust, reproducible, and automated workflows suitable for production use.
• Create modular, testable components and establish clear versioning to ensure reproducibility across environments.
• Design and maintain data models, storage solutions, and metadata catalogues that support efficient querying and lineage tracking.
• Implement and enforce data governance practices, including data classification, retention policies, and access control frameworks.
• Maintain comprehensive lineage tracking (e.g., with OpenLineage or equivalent) and ensure auditability of all datasets.
• Develop orchestration and scheduling frameworks to automate both operational and R&D pipelines.
• Implement observability practices — monitoring, alerting, and automated recovery — to ensure high reliability and performance.
• Drive continuous improvement in efficiency, scalability, and cost optimisation of data workflows across AWS/GCP/Azure.
• Embed security-by-design principles into all data handling, including encryption, authentication, and secrets management.
• Ensure all pipelines and data stores comply with regulatory requirements relevant to diagnostics and healthcare (e.g., ISO27001, ISO13485, CLIA/CAP, GDPR).
• Contribute to technical documentation and evidence for audits and certification processes.
• Partner with computational biologists and product engineers to define data requirements and shape infrastructure decisions.
• Provide technical mentorship and guidance to team members on data engineering best practices.
• Document systems and processes through runbooks, design specifications, and operational guides.
• Contribute to code reviews, internal knowledge-sharing sessions, and cross-functional project planning.
• Evaluate and integrate new technologies to improve data processing, observability, and scalability.
• Identify and remove bottlenecks in the data lifecycle — from ingestion to reporting — to accelerate insight generation.
• Support the adoption of modern DevOps and MLOps approaches for scientific and product data pipelines.

Data Engineer

Scientific Data Engineer