Senior Big Data Engineer

S&P Global

Senior Big Data Engineer designing and developing data solutions at S&P Global. Building ETL pipelines and supporting cloud-based data applications for strategic business outcomes.

Posted 5/16/2026full-timeNew York City • New York • 🇺🇸 United StatesSenior💰 $75,671 - $150,936 per yearWebsite

Tech Stack

Tools & technologies

ApacheAWSAzureCloudDockerETLGoogle Cloud PlatformHadoopInformaticaJavaKafkaKubernetesMapReduceOpenShiftPulsarPythonScalaSparkSQL

About the role

Key responsibilities & impact

Design and develop scalable data processing pipelines using distributed computing frameworks to handle large-scale enterprise datasets
Build and maintain ETL processes that extract, transform, and load data into strategic information products supporting organizational goals
Develop data-intensive applications and services using modern programming languages and cloud-based analytics platforms
Implement stream processing solutions using real-time data processing technologies to support business-critical operations
Collaborate with cross-functional teams to translate complex business requirements into robust technical solutions
Provide production support and troubleshooting for data systems, ensuring high availability and performance

Requirements

What you’ll need

6+ years of hands-on experience in technology application development and production support
5+ years of experience developing data pipelines that extract, transform, and load data using programming languages such as Python, Scala, or Java
Minimum 3+ years of experience developing and supporting ETL processes using cloud-based analytics platforms (such as Databricks, Snowflake, or Azure Synapse)
Experience building data-intensive applications using modern programming technologies (including but not limited to C#, Java, Python, Scala, and SQL)
Proficiency with cloud computing environments (such as AWS, Azure, or Google Cloud Platform)
Experience with stream processing technologies (such as Apache Kafka, Apache Pulsar, or Amazon Kinesis) for real-time data processing (preferred)
Hands-on experience with distributed processing frameworks (such as Apache Spark, Dask, or Hadoop MapReduce) for large-scale data analytics (preferred)
Knowledge of data integration tools (such as Apache NiFi, Talend, or Informatica) and database replication technologies (preferred)
Experience with CI/CD pipelines, containerization technologies (such as Docker, Podman, or containerd), and orchestration platforms (such as Kubernetes, Docker Swarm, or OpenShift) (preferred)

Benefits

Comp & perks

Health & Wellness: Health care coverage designed for the mind and body.
Flexible Downtime: Generous time off helps keep you energized for your time on.
Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data processing pipelinesETL processesdata-intensive applicationsstream processing solutionsPythonScalaJavaSQLcloud computingdistributed processing frameworks

Soft Skills

collaborationtroubleshootingproduction supportcommunicationproblem-solving