Scientific Data Architect
TetraScience
AWSCloudPython
Data Engineer
Olive Jar Digital
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇬🇧 United Kingdom
Visit company websiteJob Level
Mid-LevelSenior
Tech Stack
AzureCloudPySparkPythonSQL
About the role
- Design, build, and optimise data pipelines for ingestion, transformation, and delivery across structured and unstructured sources.
- Develop and maintain metadata capture processes to support data cataloguing and lineage.
- Implement and document data quality rules, retention policies, and data optimisation processes.
- Support large-scale deployment and configuration of Microsoft Purview.
- Establish automated scanning and metadata ingestion for source systems.
- Configure data lineage, glossary, and metadata structures to align with governance and compliance standards (UK GDPR, ISO 27001).
- Assess current data platform performance; identify and resolve bottlenecks across ingestion, processing, query, and reporting layers.
- Conduct benchmarking (e.g., latency, throughput, cost efficiency) and implement storage/DQ optimisations.
- Develop and apply storage tiering and retention strategies.
- Collaborate with Data Governance and Data Quality teams to ensure consistency, accuracy, and compliance.
- Contribute to documentation and knowledge transfer for data engineering and digital services teams.
- Participate in agile delivery ceremonies, workshops, and technical review sessions.
Requirements
- Strong experience in data engineering within cloud environments, ideally Azure.
- Proficiency in data pipeline development (e.g., Azure Data Factory, Databricks, Synapse, or similar).
- Experience implementing and managing Microsoft Purview or equivalent data cataloguing tools.
- Solid understanding of metadata management, data lineage, and data quality frameworks.
- Knowledge of data governance principles, including GDPR compliance and data retention policy design.
- Strong experience in SQL, Python, or PySpark for data manipulation and integration.
- Proven ability to optimise performance and manage large data volumes efficiently.
- Experience working in agile delivery environments with cross-functional data teams.
Benefits
- Innovative Culture: Join a team that thrives on creativity and embraces innovation.
- Collaborative Team: At Olive Jar Digital, collaboration is at the heart of everything we do.
- Professional Growth: We are dedicated to the growth and development of our employees.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data pipeline developmentSQLPythonPySparkmetadata managementdata lineagedata quality frameworksstorage tieringdata optimisationbenchmarking
Soft skills
collaborationdocumentationknowledge transferproblem solvingcommunication
