Data Engineer

• Design and architect comprehensive data center infrastructure solutions, including network, storage, compute, and security components
• Develop and communicate technical strategy and roadmap for data center projects, ensuring alignment with organizational objectives
• Lead project planning, execution, and delivery, managing timelines, budgets, and resource allocation effectively
• Oversee cross-functional teams including engineers, vendors, and stakeholders to ensure successful project outcomes
• Conduct detailed technical assessments and feasibility studies to evaluate data center requirements and optimization opportunities
• Ensure all data center designs comply with industry standards, regulatory requirements, and best practices
• Mentor and guide technical teams, fostering a culture of continuous learning and technical excellence
• Analyze and optimize data center performance, efficiency, and cost-effectiveness through innovative solutions
• Evaluate emerging technologies and recommend adoption strategies for data center modernization
• Manage vendor relationships and evaluate third-party solutions for data center infrastructure components

Principal Architect – Data Center Projects

• Manage and maintain environmental data within EarthSoft EQuIS and related systems.
• Import, validate, QA/QC, and manage laboratory, field, historical, and monitoring data.
• Configure EQuIS workflows, formats, reports, and standard data management processes.
• Develop dashboards, reports, and visualizations using Power BI, EQuIS Enterprise, and related tools.
• Support project teams with data queries, reporting requests, and data deliverable preparation.
• Support data migration of historical data into enterprise systems, ensuring compatibility and formatting standards.
• Coordinate with laboratories and field teams to ensure timely receipt and processing of environmental data.
• Perform data quality reviews and identify inconsistencies, anomalies, and compliance issues.
• Support implementation of EQuIS modules such as Collect, SPM, Enterprise, and Professional.
• Contribute to workflow automation, process improvements, and standardization initiatives.
• Manage multiple assignments while meeting schedule, quality, and budget expectations.

Senior Associate, Data Management – Data Migration, Verification, EQuIS, SQL

• Design, develop, and maintain large-scale data systems
• Develop and implement ETL processes using various tools and technologies
• Collaborate with cross-functional teams to design and implement data models
• Work with big data tools like Hadoop, Spark, PySpark, and Kafka
• Develop scalable and efficient data pipelines
• Troubleshoot data-related issues and optimize data systems
• Transition and upskill into Databricks & AI/ML projects

• Design, develop, and optimize data pipelines using Azure Data Factory (ADF).
• Migrate enterprise data workloads to Microsoft Azure using cloud-native services.
• Develop and maintain solutions using Azure SQL Database and Databricks.
• Implement secure data migration strategies with minimal downtime.
• Configure and manage Managed Identity for secure access to Azure resources.
• Implement data encryption, data masking, and security best practices to protect sensitive information.
• Optimize data storage, transformation, and processing for performance and scalability.
• Develop backup and recovery strategies for Azure data services.
• Monitor, troubleshoot, and optimize Azure data platforms.
• Collaborate with cross-functional teams to deliver scalable cloud data solutions.

• Design, develop, and maintain scalable cloud-based data pipelines using Snowflake and modern ELT tools.
• Build and manage data ingestion pipelines using Fivetran, Airbyte, or similar integration platforms.
• Develop modular, reusable, and well-documented data transformation models using dbt (Data Build Tool).
• Design and optimize dimensional data models (Star Schema and Snowflake Schema) for analytics and reporting.
• Develop incremental data loading strategies and Change Data Capture (CDC) processes.
• Optimize Snowflake performance through clustering, partitioning, query tuning, and warehouse optimization.
• Implement data quality checks, validation rules, and monitoring to ensure data accuracy and reliability.
• Collaborate with business analysts, data architects, and stakeholders to translate business requirements into technical solutions.
• Integrate data from ERP, CRM, eCommerce, finance, and third-party SaaS applications.
• Troubleshoot pipeline failures, resolve performance bottlenecks, and support production environments.
• Implement CI/CD processes for dbt projects and data pipelines.
• Create and maintain technical documentation, data lineage, and metadata.

Data Engineer – Snowflake, DBT

• Support enterprise AI and Generative AI initiatives by building scalable, secure, and AI-ready data platforms.
• Partner with Data Engineers, AI Engineers, Data Scientists, and business stakeholders to develop high-quality data pipelines.
• Enable Retrieval-Augmented Generation (RAG) solutions and prepare enterprise data for AI-powered applications.
• Design, develop, and maintain scalable data pipelines using Azure Databricks, Snowflake, and Azure Data Factory.
• Build ELT/ETL pipelines to ingest data from ERP, CRM, manufacturing, supply chain, and other enterprise applications.
• Develop AI-ready datasets that support machine learning and Generative AI use cases.
• Integrate structured, semi-structured, and unstructured data into centralized data platforms.
• Collaborate with AI Engineers to expose enterprise data securely to LLM-powered applications.
• Optimize data models, transformations, and query performance for analytics and AI workloads.
• Implement data quality, governance, lineage, monitoring, and security best practices.
• Build reusable data transformation frameworks using Python, SQL, Spark, and dbt (if applicable).
• Develop REST API integrations to support AI services and enterprise applications.
• Participate in architecture discussions, code reviews, CI/CD implementations, and Agile ceremonies.
• Support production deployments, monitoring, troubleshooting, and performance tuning.

Data Engineer – AI

• Design and develop robust, highly scalable, and efficient ELK (Elasticsearch, Logstash, Kibana) stack solutions hosted on AWS Cloud.
• Evaluate and optimize Elasticsearch clusters, including configuration parameters, indexing strategies, search, relevancy tuning, query performance tuning, security, scalability, and administration.
• Design data structures and models tailored for Elasticsearch.
• Develop complex queries with joins across large datasets using both RDBMS and Elasticsearch.
• Build data ingestion pipelines integrating Elasticsearch with various sources via Logstash, MQ via IBM Integration Bus/ACE , and Java based batch processes scheduled on AWS EC2.
• Develop and expose RESTful APIs using Java Spring Boot for applications hosted on AWS Lambda.
• Write clean, high-quality, and maintainable Java code using the Spring Boot framework.
• Collaborate with Integration and Architecture teams to design, document, build, secure, and maintain ELK based solutions.
• Deploy applications utilizing AWS services such as EC2, S3, Lambda, Step functions, IAM, SNS, API Gateway, VPC, Transit Gateway, and Cloud Watch.
• Manage CI/CD pipelines and DevOps process with using Jenkins and GitHub Actions.
• Oversee build and release process with tools like Maven and Jenkins.
• Actively participate in Agile ceremonies, providing status updates and technical insights during sprint meetings.
• Work cross-functionally to coordinate efforts and deliver projects on time.
• Troubleshoot and resolve production issues related to search, indexing, and backend performance.
• Stay current with industry trends and best practices related to Elasticsearch, Java development and cloud-based architectures.

Data Engineer – Elastic

• Assist in building, testing, and maintaining data pipelines under the guidance of senior data engineers.
• Work with structured and unstructured data formats such as CSV, JSON, and XML.
• Learn and apply basic concepts of databases (SQL/NoSQL) for data querying and management.
• Support in data transformation and integration tasks using Python or other scripting languages.
• Gain exposure to **Azure fundamentals** and basic cloud storage concepts.
• Collaborate with team members on academic-style projects, POCs, and small-scale data solutions.
• Document learnings, processes, and workflows to support knowledge sharing.
• Continuously develop technical and problem-solving skills through mentorship and hands-on practice.

Trainee Data Engineer

• Design, develop, and optimize scalable data pipelines using Azure Synapse Analytics, Spark Pools, Azure Data Factory (ADF), and Azure Data Lake.
• Develop high-performance solutions using SQL and Python, ensuring scalability, maintainability, and optimized query performance.
• Build and manage modern data platforms on Microsoft Azure, supporting structured and unstructured data ingestion at enterprise scale.
• Design and implement robust data models, including dimensional modelling and Slowly Changing Dimensions (SCD), to support reporting and analytics.
• Optimize data partitioning, indexing, storage, versioning, and compute costs for large-scale cloud workloads.
• Implement CI/CD pipelines and adopt DataOps best practices for automated deployment, testing, and monitoring.
• Ensure data quality through unit testing, validation frameworks, and automated quality checks.
• Collaborate directly with clients, business stakeholders, architects, and cross-functional teams to gather requirements, design solutions, and provide technical guidance.
• Troubleshoot production issues, perform root cause analysis, and recommend performance improvements.
• Mentor junior engineers by promoting engineering best practices, code quality, and technical excellence.

Senior Data Engineer

• Design and deliver robust data solutions that enable AI and advanced analytics across the organization.
• Own the end-to-end data lifecycle: from requirement gathering and data mapping to implementing scalable pipelines.
• Ingest data from disparate engineering and business tools into a single, curated, well-structured data lake and data catalogue.
• Clean and curate complex datasets. Develop and maintain data catalogues, metadata, and data mapping documentation.
• Architect, implement, and optimize scalable ETL/ELT processes, moving data from diverse sources.
• Leverage Python to automate data validation, monitoring, and repetitive workflows.
• Design and implement dashboards and reports using platforms like Power BI or Tableau to provide actionable insights.

Data Engineer, 10-15 years

• Lead the design, development, and maintenance of robust, scalable data pipelines for ingestion, transformation, and processing of large datasets in an on-premises environment.
• Own architectural and design decisions for data solutions, evaluating trade-offs and defining technical standards for the team.
• Mentor, guide, and support other data engineers through code reviews, design reviews, technical coaching, and hands-on problem-solving.
• Build and optimize data workflows using Python, Spark, and the Hadoop ecosystem.
• Work extensively with Hadoop ecosystem components (Hive, HDFS, Impala) to manage and query large-scale data.
• Manage and optimize batch scheduling and job orchestration using enterprise schedulers such as CA7 or Control-M.
• Ensure data quality, integrity, and performance across data platforms.
• Collaborate with data analysts, data scientists, and business stakeholders to translate data requirements into sound technical designs.
• Troubleshoot and resolve complex issues in data pipelines and production environments, acting as an escalation point for the team.
• Champion best practices for coding standards, version control, testing, and documentation.
• Stay current with emerging technologies, particularly AI/ML capabilities, and identify opportunities to apply them to data engineering workflows.

Big Data Engineer

• Design, build, and optimize scalable data pipelines and ETL/ELT workflows for large, complex datasets.
• Design and implement foundational data architecture supporting identity resolution and ID graph systems.
• Develop and enhance systems supporting identity resolution and ID graph construction (data ingestion, normalization, matching, and deduplication).
• Process and unify multi-source datasets (cookies, device IDs, behavioral data, third-party and proprietary data).
• Write efficient, testable, and maintainable code using Python and SQL for large-scale data processing.
• Optimize data models, queries, and storage strategies for performance, scalability, and cost efficiency.
• Build and maintain data validation, monitoring, and alerting systems to ensure data quality and reliability.
• Troubleshoot, debug, and improve existing data pipelines and infrastructure.
• Take ownership of complex data problems end-to-end, from initial design through production deployment.
• Contribute to key technical decisions related to data architecture, scalability, and system design.
• Collaborate with data, platform, DevOps, and product teams to deliver scalable, production-ready solutions.
• Translate business and product requirements into practical, performant data solutions.
• Document data pipelines, systems, and workflows clearly.
• Continuously improve system performance, data quality, and pipeline resilience.
• Contribute to building new capabilities that improve how customers understand and leverage data insights.

Senior GCP Data Engineer

• Work in collaborative environment including pairing and mobbing with other cross-functional engineers
• Work on a small agile team to deliver working, tested software
• Work effectively with fellow data engineers, product owners, data champions and other technical experts
• Demonstrate technical knowledge/leadership skills and advocate for technical excellence
• Develop exceptional Analytics data products using streaming, batch ingestion patterns in the Google Cloud Platform with solid Datawarehouse principles
• Be the Subject Matter Expert in Data Engineering and GCP tool technologies

• Design, develop, and maintain scalable data pipelines using Microsoft Azure Fabric, Databricks, and Azure data services.
• Build and optimize robust ETL/ELT processes for structured, semi-structured, and unstructured data across enterprise environments.
• Collaborate with business stakeholders to understand data requirements and translate them into scalable technical solutions.
• Develop and maintain reliable data integration workflows that ensure data accuracy, consistency, and availability.
• Optimize data processing pipelines for performance, scalability, cost efficiency, and operational reliability.
• Leverage AI-assisted development tools and modern engineering practices to accelerate solution delivery and improve code quality.
• Monitor, troubleshoot, and resolve production data pipeline issues while proactively identifying opportunities for automation and optimization.
• Work closely with architects, analysts, developers, and cross-functional teams throughout the project lifecycle to deliver high-quality data solutions.
• Implement best practices for data engineering, governance, documentation, testing, and operational support.
• Take ownership of project deliverables by ensuring quality, meeting timelines, communicating risks proactively, and continuously improving engineering processes.

• Own reliability and performance of operational pipelines across our product catalog infrastructure
• Identify and reduce technical debt
• Build and maintain low latency data APIs
• Implement CDC patterns to keep catalog data synchronized
• Build monitoring, observability, and automated testing
• Translate business requirements from non-technical stakeholders into durable pipeline logic
• Own code versioning, deployment, and incident response

Software Engineer II, Data Engineering

Senior Data Engineer, Microsoft Fabric Engineer

• Design, develop, and operate scalable and maintainable data pipelines in the Azure Databricks environment
• Develop all technical artefacts as code, implemented in professional IDEs, with full version control and CI/CD automation
• Enable data-driven decision-making in Supply Chain Management (SCM) by ensuring high data availability, quality, and reliability
• Implement data products and analytical assets using software engineering principles in close alignment with business domains and functional IT
• Apply rigorous software engineering practices such as modular design, test-driven development, and artifact reuse in all implementations
• Global delivery footprint; cross-functional data engineering support across SCM domains
• Collaboration with business stakeholders, functional IT partners, product owners, architects, ML/AI engineers, and Power BI developers
• Agile, product-team structure embedded in an enterprise-scale Azure environment
• Design scalable batch and streaming pipelines in Azure Databricks using PySpark and/or Scala
• Implement ingestion from structured and semi-structured sources (e.g., SAP, APIs, flat files)
• Build bronze/silver/gold data layers following the defined lakehouse layering architecture & governance
• Implement use-case driven dimensional models (star/snowflake schema) tailored to SCM needs
• Ensure compatibility with reporting tools (e.g., Power BI) via curated data marts and semantic models
• Implement enterprise-level data warehouse models (domain-driven 3NF models) for SCM data, closely aligned with data engineers for other business domains
• Develop and apply master data management strategies (e.g., Slowly Changing Dimensions)
• Develop automated data validation tests using frameworks
• Monitor pipeline health, identify anomalies, and implement quality thresholds
• Establish data quality transparency by defining and implementing meaningful data quality rules with source system and business stakeholders and implementing related reports
• Develop and structure pipelines using modular, reusable code in a professional IDE
• Apply test-driven development (TDD) principles with automated unit, integration, and validation tests
• Integrate tests into CI/CD pipelines to enable fail-fast deployment strategies
• Commit all artifacts to version control with peer review and CI/CD integration
• Work closely with Product Owners to refine user stories and define acceptance criteria
• Translate business requirements into data contracts and technical specifications
• Participate in agile events such as sprint planning, reviews, and retrospectives
• Document pipeline logic, data contracts, and technical decisions in markdown or auto-generated docs from code
• Align designs with governance and metadata standards (e.g., Unity Catalog)
• Track lineage and audit trails through integrated tooling
• Profile and tune data transformation performance
• Reduce job execution times and optimize cluster resource usage
• Refactor legacy pipelines or inefficient transformations to improve scalability

IT Engineer, Data Lakehouse

• Design and implement ML models including regression, classification, clustering, and optimization algorithms.
• Build and deploy GenAI applications using LLMs for tasks such as summarization, sentiment analysis, and content generation.
• Collaborate with cross-functional teams to identify opportunities for AI integration into business processes.
• Conduct research on emerging GenAI technologies and frameworks (e.g., LangChain, LlamaIndex).
• Fine-tune and optimize models for performance, scalability, and accuracy.
• Develop and maintain robust data pipelines and workflows.
• Stay updated with the latest advancements in AI, ML, and NLP.

Associate Data Scientist, AI Engineering

• Design, develop, and optimize data structures within Snowflake on Azure
• Build and manage complex ETL/ELT data pipelines using Azure Data Factory (ADF)
• Work proficiently within the Azure data ecosystem
• Develop complex SQL logic and perform query optimization
• Implement Snowflake security configurations
• Manage CI/CD pipelines using Azure DevOps
• Assist in developing automated testing frameworks for data pipeline validation
• Apply dimensional modeling concepts (Star and Snowflake schemas) to build efficient data structures
• Support the integration of financial data from platforms like Sage Intacct GL, SkyGen, and ClearWater
• Lead, mentor, and conduct code reviews for Junior Data Engineers.

• Design and develop data warehouse solutions using the Snowflake Data Platform on Azure
• Lead the development of data pipelines using Azure Data Factory (ADF)
• Implement incremental data loading using CDC and watermark-based methods
• Apply best practices for Snowflake optimization and security
• Translate detailed business requirements into technical data solutions
• Implement CI/CD pipelines using Azure DevOps
• Lead requirement gathering with stakeholders and assist in developing automated testing for data pipelines and data quality

Data Engineer jobs in India

See all jobs on JobTailor

Principal Architect – Data Center Projects

Senior Associate, Data Management – Data Migration, Verification, EQuIS, SQL

Data Engineer

Data Engineer

Data Engineer – Snowflake, DBT

Data Engineer – AI

Data Engineer – Elastic

Trainee Data Engineer

Senior Data Engineer

Data Engineer, 10-15 years

Big Data Engineer

Senior GCP Data Engineer

Data Engineer

Senior Data Engineer

Software Engineer II, Data Engineering

Senior Data Engineer, Microsoft Fabric Engineer

IT Engineer, Data Lakehouse

Associate Data Scientist, AI Engineering

Senior Data Engineer

Associate Architect – Data