Data Engineer

EXL

Data and AI Engineer modernizing legacy Spark and Hadoop infrastructure within AWS and Databricks. Building AI agents, LLM workflows, and optimized pipelines for business intelligence and AI use cases.

Posted 9/10/2026full-timeBengaluru • 🇮🇳 IndiaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and optimizing AI-oriented pipeline architectures, leveraging tools such as Databricks and AWS for efficient data processing and workflow automation. Proficient in integrating LLM APIs and utilizing orchestration frameworks to enhance production AI workloads.

Highest-signal resume keywords

Apache SparkPython (PySpark)AWS (S3, IAM)DatabricksLLM Integration (Anthropic Claude)

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

SQL QueryingData Processing OptimizationWorkflow AutomationReal-Time Feature TokenizationMetadata TaggingData Pipeline ArchitectureDebuggingData Extract OptimizationOut Of-Memory (OOM) Error ResolutionACID Transactions

Tools & Technologies

Databricks Mosaic AILangChainDelta LakeTableauLookerPower BI

Industry Keywords

Big DataAI AgentsUnstructured Data IngestionOrchestration FrameworksBI Tools

Tech Stack

Tools & technologies

ApacheAWSHadoopMapReducePySparkPythonScalaSparkSQLTableau

About the role

Key responsibilities & impact

Deconstruct legacy Apache Spark and Hadoop MapReduce workflows and re-architect them as optimized, production-ready pipelines within AWS and Databricks
Design, build, and deploy intelligent AI agents and workflow automation tools using LLMs such as Anthropic Claude
Build and optimize AI-oriented pipeline architectures for unstructured data ingestion, real-time feature tokenization, and metadata tagging for vector databases
Monitor, maintain, and troubleshoot existing Hadoop big data workloads during the multi-phase migration
Resolve data-processing bottlenecks and Out Of-Memory (OOM) errors
Support downstream BI stakeholders through functional enhancements, bug fixes, and data extract optimizations
Use Databricks and Delta Lake features, including ACID transactions, Z-Ordering, and caching, to improve performance, reliability, and cost efficiency
Leverage Databricks Vector Search, Mosaic AI, and Lakeflow to orchestrate, track, and serve production AI workloads

Requirements

What you’ll need

5-8 years of experience
Hands-on experience or deep technical familiarity building functional AI agents
Experience integrating LLM APIs, specifically Anthropic Claude
Experience with orchestration frameworks such as LangChain or Databricks Mosaic AI Agent Framework
Foundational understanding of distributed storage and computing concepts, including partitioning, shuffling, caching, and broadcast joins
Solid hands-on experience with Apache Spark
Strong proficiency in Python (PySpark) or Scala
Intermediate-to-advanced SQL querying capabilities, including window functions, query tuning, and complex joins
Direct experience or deep theoretical knowledge of AWS, including S3 and IAM
Experience with Databricks environments
Practical experience with a BI tool such as Tableau, Looker, or Power BI
Ability to debug calculated fields, modify parameters, and troubleshoot slow-loading reports

Benefits

Comp & perks

🌐 Worldwide ❌ Jobs You've Hidden ⭐️ Saved Jobs ✅ Applied Jobs ✉️ Email Alerts 👤 Account EXL Website LinkedIn All Job Openings 10,000+ employees 🏥 Healthcare 🛡️ Insurance 📦 Logistics 💰 $2M Venture Round on 2015-01 Healthcare
Insurance
Logistics EXL is a business consulting and services firm that focuses on leveraging data to enhance business operations and decision-making. With a strong emphasis on collaboration and adaptability, EXL partners with organizations to address their unique needs and culture while integrating data science and technology solutions. The company's areas of expertise include operations management, decision analytics, digital transformation, and various industries such as healthcare, finance, and insurance. EXL's mission is to help clients drive business evolution and maintain competitive advantage through tailored solutions and effective use of data. Data Engineer 🔥 2 hours ago 🏢🏡 Bengaluru – Hybrid ⏰ Full Time 🟡 Mid-level 🟠 Senior 🚰 Data Engineer 👻 Ghost score 11% Apache AWS Hadoop MapReduce PySpark Python Scala Spark SQL Tableau Apply Now Customize resume + cover letter Report problem ☆ Save ☑️ Mark as applied ❌ Hide 📋 Description
Deconstruct legacy Apache Spark and Hadoop MapReduce workflows and re-architect them as optimized, production-ready pipelines within AWS and Databricks
Design, build, and deploy intelligent AI agents and workflow automation tools using LLMs such as Anthropic Claude
Build and optimize AI-oriented pipeline architectures for unstructured data ingestion, real-time feature tokenization, and metadata tagging for vector databases
Monitor, maintain, and troubleshoot existing Hadoop big data workloads during the multi-phase migration
Resolve data-processing bottlenecks and Out Of-Memory (OOM) errors
Support downstream BI stakeholders through functional enhancements, bug fixes, and data extract optimizations
Use Databricks and Delta Lake features, including ACID transactions, Z-Ordering, and caching, to improve performance, reliability, and cost efficiency
Leverage Databricks Vector Search, Mosaic AI, and Lakeflow to orchestrate, track, and serve production AI workloads 🎯 Requirements
5-8 years of experience
Hands-on experience or deep technical familiarity building functional AI agents
Experience integrating LLM APIs, specifically Anthropic Claude
Experience with orchestration frameworks such as LangChain or Databricks Mosaic AI Agent Framework
Foundational understanding of distributed storage and computing concepts, including partitioning, shuffling, caching, and broadcast joins
Solid hands-on experience with Apache Spark
Strong proficiency in Python (PySpark) or Scala
Intermediate-to-advanced SQL querying capabilities, including window functions, query tuning, and complex joins
Direct experience or deep theoretical knowledge of AWS, including S3 and IAM
Experience with Databricks environments
Practical experience with a BI tool such as Tableau, Looker, or Power BI
Ability to debug calculated fields, modify parameters, and troubleshoot slow-loading reports Apply Now 📊 Check your resume score for this job Improve your chances of getting an interview by checking your resume score before you apply. Check Resume Score Similar Jobs Lead Data Engineer, AI 🔥 12 hours ago Enable Data 51 - 200 💼 Consulting 🏥 Healthcare 🏭 Manufacturing Website LinkedIn All Job Openings Lead Data Engineer building Databricks data pipelines, AI applications, and Azure CI/CD. Developing Claude-based interfaces and enterprise data solutions for Enable Data. 🏢🏡 Bengaluru – Hybrid ⏰ Full Time 🟠 Senior 🚰 Data Engineer Azure Cloud React Unity Technical Data Engineer 🕒 August 28 Mouser Electronics 1001 - 5000 💼 Consulting 📦 Logistics 📣 Marketing Website LinkedIn All Job Openings Technical Data Engineer maintaining PIM data and quality for electronic components at Mouser Electronics. Supporting engineers sourcing products through research, classification, normalization, and taxonomy improvements. 🏢🏡 Bengaluru – Hybrid ⏰ Full Time 🟡 Mid-level 🟠 Senior 🚰 Data Engineer Data Engineer 🕒 August 13 Serko Ltd. 201 - 500 💼 Consulting 📦 Logistics ✈️ Travel Website LinkedIn All Job Openings Data Engineer scaling Serko’s global business travel and expense data platform in Bengaluru. Building reliable pipelines, models, and cloud warehouse solutions with dbt, Snowflake, BigQuery, SQL, and Python. 🏢🏡 Bengaluru – Hybrid ⏰ Full Time 🟡 Mid-level 🟠 Senior 🚰 Data Engineer Azure BigQuery Cloud MongoDB NoSQL Python SQL Vault Data Engineer II 🕒 August 11 HackerRank 201 - 500 🎯 Recruiter 📚 Education ☁️ SaaS Website LinkedIn All Job Openings Data Engineer II building StarRocks, Hudi, Trino, and Spark pipelines for HackerRank’s developer hiring platform. Preparing AI-ready datasets and supporting analytics, exports, and natural-language querying features. 🏢🏡 Bengaluru – Hybrid 💰 $60M Series D on 2022-03 ⏰ Full Time 🟢 Junior 🟡 Mid-level 🚰 Data Engineer 🚫👨‍🎓 No degree required Apache AWS Python Spark SQL Senior Data Engineer 🕒 August 11 Serko Ltd. 201 - 500 💼 Consulting 📦 Logistics ✈️ Travel Website LinkedIn All Job Openings Senior Data Engineer scaling Serko’s global business travel and expense technology data platform. Building pipelines, models, warehouses, and governed solutions from Bengaluru. 🏢🏡 Bengaluru – Hybrid ⏰ Full Time 🟠 Senior 🚰 Data Engineer Azure BigQuery Cloud MongoDB NoSQL Python SQL Vault View More Data Engineer Jobs 🌐 Worldwide Built by Lior Neu-ner. I'd love to hear your feedback — Get in touch via DM or support@remoterocketship.com Search Remote jobs Search Jobs by country Search jobs by city Search jobs by job title Search entry-level jobs Search junior-level jobs Search senior-level jobs Search jobs by tech stack Search jobs by contract type Search remote internships Search remote part-time jobs Remote jobs Anywhere in the World Companies Hiring Anywhere in the World Companies Hiring Sales People Anywhere in the World Companies Hiring Software Engineers Anywhere in the World Resources About us Advice Tips for finding remote jobs Interview questions and answers Resume examples Cover letter examples Post a job Affiliates Is Remote Rocketship legit? Privacy policy Terms of service Job board SEO course Remote Job Search MasterClass Resume Review AI Apply Copilot OpenClaw job finder API docs Find jobs using your resume Jobs by Country Remote jobs anywhere in the world (Worldwide remote jobs) Remote jobs United States Remote jobs Australia Remote jobs Brazil Remote jobs Canada Remote jobs France Remote jobs Ireland Remote jobs Germany Remote jobs Netherlands Remote jobs Spain Remote jobs UK Popular Jobs Remote data analyst jobs Remote customer support jobs Remote executive assistant jobs Remote marketing jobs Remote product designer jobs Remote product manager jobs Remote project manager jobs Remote recruiter jobs Remote sales jobs Remote software engineer jobs Jobs by Type Remote full-time jobs Remote part-time jobs Remote contract jobs Remote internship jobs Remote entry-level jobs Remote jobs with no experience required Remote junior jobs (1-3 years of experience) Digital nomad jobs Remote jobs with no degree required Freelance remote jobs Temporary remote jobs Remote jobs hiring now Stay at home mom jobs