Data Engineer – Datacore, IOTT

Information Commissioner's Office

Data Engineer delivering data solutions for batch and streaming use cases on the Datacore/IOTT platform. Integrating industrial, IoT, and enterprise data sources into a secure, scalable cloud platform.

Posted 5/22/2026part-timeLincoln • 🇬🇧 United KingdomMid-LevelSenior💰 £45,000 per yearWebsite

Tech Stack

Tools & technologies

AzureCloudIoTKafkaPythonSparkSQL

About the role

Key responsibilities & impact

The Data Engineer (Datacore / IOTT) will have the core skills to carry out the following activities: Design robust, secure, and supportable data solutions for both batch and streaming use cases, ensuring scalability, resilience, and security by design
Develop, test, and support data pipelines integrating data from industrial, IoT, and enterprise systems using technologies such as Azure Synapse, and event streaming platforms (e.g. Event Hubs, Kafka, MQTT, Quix)
Work with edge and integration platforms (e.g. HighByte, Ignition, Quix, or similar) to ingest, model, and standardise operational data, aligned to a Unified Namespace (UNS) approach for organising and distributing real-time operational data
Build and optimise solutions for handling time-series and telemetry data, ensuring performance and usability at scale
Collaborate with architects, cloud engineers, and platform teams to design and evolve data platform capabilities, including CI/CD, deployment automation, and environment management
Ensure data quality, observability, and reliability across real-time and batch pipelines
Where required, contribute to downstream data modelling and analytics to support reporting and insight
Build automated test frameworks and develop test harnesses for each component, increasing overall test coverage and ensuring a culture of quality and reliability across the platform.

Requirements

What you’ll need

Strong experience in data engineering using cloud platforms (e.g. Azure), including tools such as Azure Synapse, Spark, Python, and SQL
Experience working with real-time or streaming data pipelines (e.g. Event Hubs, Kafka, MQTT, Quix, or similar)
Experience working with time-series or telemetry data
Experience integrating data from APIs, industrial systems, or edge platforms (e.g. HighByte, Ignition, or similar)
Familiarity with Unified Namespace (UNS) principles and MQTT-based architecture is advantageous
Experience in data modelling (e.g. relational modelling, dimensional modelling, or similar approaches suited to operational systems)
Experience delivering end-to-end data solutions from ingestion through to production deployment
Experience working in an Agile/Scrum environment
Experience collaborating with platform and engineering teams to support CI/CD, DevOps, and engineering best practices
Knowledge of Azure DevOps or similar tools
Awareness of security, data governance, and working within regulated or operationally critical environments is desirable.

Benefits

Comp & perks

Personal private health care
26 days annual leave – rising with length of service
Flexible working
Pension scheme – Anglian Water double-matches your contributions up to 7%
Bonus scheme
Flexible benefits to support your wellbeing and lifestyle.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data engineeringAzure SynapseSparkPythonSQLreal-time data pipelinesdata modellingend-to-end data solutionsAgileDevOps

Soft Skills

collaborationcommunicationproblem-solvingquality assurancereliability focus