Codvo.ai

Data Engineer Lead, OT Data, Oil & Gas

Codvo.ai

full-time

Posted on:

Location Type: Remote

Location: India

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AirflowApacheAzureCloudETLIoTNoSQLPySparkPythonSparkSQL

About the role

  • Architect & Build Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems and ETL/ELT pipelines.
  • Integrate Diverse Data Sources: Develop processes to ingest and integrate high-volume, high-velocity data from SCADA systems, historians, DCS, PLC, and IoT sensors.
  • Cloud Data Platform Development: Implement and manage data solutions on Microsoft Azure, leveraging services like Azure IoT Hub.
  • Data Modelling & Warehousing: Design and implement data models optimized for time-series data from industrial assets.
  • Enable Advanced AI: Build the data infrastructure to support AI/ML models for predictive maintenance and anomaly detection.
  • Champion Master Data Management (MDM): Create a single source of truth for critical data domains.
  • Ensure Data Quality & Governance: Implement data quality checks and monitoring to ensure accuracy and consistency.
  • Embrace Industry Standards: Implement industry-specific standards for interoperability across sectors.
  • Collaborate & Innovate: Work with cross-functional teams to understand data needs and deliver effective solutions.
  • Automate & Optimize: Identify automation opportunities to improve data delivery and scalability.
  • Security First: Maintain security best practices to protect sensitive data assets.

Requirements

  • 6+ years of experience in data engineering
  • Bachelor's in engineering, Information Systems, or a related quantitative field
  • Experience within oil and gas industry is highly preferred
  • Demonstrable experience building and operationalizing large-scale data pipelines and applications
  • Expert-level proficiency in SQL and Python for data manipulation and pipeline development
  • Hands-on experience with distributed computing frameworks like Apache Spark (PySpark)
  • Deep experience with Microsoft Azure (Azure Data Lake Storage, Azure Data Factory, Azure Databricks, Azure Synapse)
  • Proven experience with modern data platforms Databricks Delta Lake
  • Understanding of machine learning lifecycles and the data requirements for training and deploying AI/ML models
  • Experience with workflow orchestration tools like Airflow, Dagster, or Azure Data Factory
  • Strong understanding of both relational and NoSQL databases
  • Proficiency with Git and CI/CD best practices
Benefits
  • Flexible remote work arrangements
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
data engineeringETLELTSQLPythonApache SparkAzure Data Lake StorageAzure Data FactoryAzure DatabricksDatabricks Delta Lake
Soft Skills
collaborationinnovationautomationoptimizationdata qualitydata governancesecurity best practices