Senior Software Engineer – Data Engineering

Caterpillar Inc.

Data Engineer creating scalable data pipelines utilizing AWS services and Snowflake. Collaborating with cross-functional teams on data requirements and solutions.

Posted 7/3/2026full-timeChennai • 🇮🇳 IndiaSeniorWebsite

Tech Stack

Tools & technologies

Amazon RedshiftAWSAzureCloudNeo4jPythonSQL

About the role

Key responsibilities & impact

Design, develop, and maintain scalable data pipelines on AWS using services such as S3, Glue, Lambda, Redshift, and EMR.
Build and optimize data warehousing solutions using Snowflake, including performance tuning and data modeling.
Write efficient and reusable code in Python and SQL for data transformation and processing.
Collaborate with cross-functional teams, including data scientists, analysts, and business stakeholders, to understand data requirements.
Develop and optimize solutions using graph databases (e.g., Neo4j, Amazon Neptune), including query design and performance tuning.
Design, build, and operate vector database solutions (e.g., Milvus, Amazon OpenSearch) to support semantic search, recommendations, RAG, and AI-driven use cases.
Integrate vector databases with LLM-based applications and AI workflows.
Monitor, troubleshoot, and improve pipeline performance and reliability.
Ensure data quality, integrity, and security across all stages of the pipeline.
Participate in code reviews, architecture discussions, and continuous improvement initiatives.

Requirements

What you’ll need

8+ years of experience in data engineering or related roles.
Strong hands-on experience with AWS cloud services, including data and AI workloads.
Deep understanding of Snowflake architecture, performance tuning, and best practices.
Advanced proficiency in Python and SQL for data pipelines, transformations, and services.
Strong understanding of graph and vector data modelling concepts and their practical applications.
Hands-on experience with graph databases (e.g., Neo4j, Neptune) and vector databases (e.g., Milvus, Amazon OpenSearch).
Experience with version control systems (e.g., Git) and Git workflows.
Experience working with Azure DevOps (AzDO) boards for backlog management in Agile environments.
Excellent analytical and problem-solving skills.
Strong communication and collaboration abilities.
Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

Benefits

Comp & perks

Professional development opportunities
Flexible work arrangements

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Data Pipeline DevelopmentData Warehousing SolutionsPerformance TuningData ModelingData TransformationSQL ProficiencyGraph Data ModelingVersion Control (Git)Agile MethodologiesData Quality Assurance

Soft Skills

Analytical SkillsProblem-Solving SkillsCommunication SkillsCollaboration Abilities