Senior Manager, Data Engineering
Capital One
Amazon RedshiftAWSAzureCassandraCloudHadoopJavaKafkaLinuxMapReduceMicroservicesMySQL+9 more
Associate Director, Data Engineering
S&P Global
full-time
Posted on:
Location Type: Hybrid
Location: New York City • New Jersey, New York • 🇺🇸 United States
Visit company websiteSalary
💰 $127,000 - $185,000 per year
Job Level
Senior
Tech Stack
AWSCloudETLPythonScala
About the role
- Lead the design and implementation of distributed data processing workloads using Python, including data access from relational databases and cloud storage technologies.
- Ensure performance, security, and reliability; coordinate with application teams to define database design.
- Collaborate with cross-functional teams to support data-driven initiatives.
- Mentor junior team members and promote best practices.
- Oversee maintenance and troubleshooting of distributed data processing systems.
- Drive innovation by evaluating and integrating new technologies.
- Produce system design documents and participate in technical walkthroughs.
- Perform application and system performance tuning and troubleshoot performance issues.
- Effectively interact with global customers, business users, and IT staff.
Requirements
- Bachelor’s degree in Computer Science, Information Systems, Engineering, or equivalent work experience.
- 10+ years of IT experience in application support or development.
- Design & build ingestion pipelines using ETL/ELT and schema-on-read in data lake technologies.
- Experience in designing transactional systems, data warehouses, data lakes, and data integrations within a big data ecosystem, particularly leveraging cloud technologies.
- Provide technical expertise in the design and implementation of data ingestion pipelines with modern AWS cloud and other technologies such as S3, Hive, Databricks, Scala, Python, and large-scale data analytics tools.
- Strong experience in managing distributed data processing workloads, including configurations.
- Experience in working with multi-threaded, high-performance, low-latency messaging systems.
- Experience in AWS cloud-based technologies.
- Experience using system tools, source control systems, utilities, and third-party products.
- Excellent communication skills, with strong verbal and writing proficiencies.
Benefits
- Health & Wellness: Health care coverage designed for the mind and body.
- Flexible Downtime: Generous time off helps keep you energized for your time on.
- Continuous Learning: Access a wealth of resources to grow your career and learn valuable new skills.
- Invest in Your Future: Secure your financial future through competitive pay, retirement planning, a continuing education program with a company-matched student loan contribution, and financial wellness programs.
- Family Friendly Perks: It’s not just about you. S&P Global has perks for your partners and little ones, too, with some best-in class benefits for families.
- Beyond the Basics: From retail discounts to referral incentive awards—small perks can make a big difference.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
PythonETLELTdata ingestion pipelinesdata lakesdata warehousesAWSScalaDatabricksperformance tuning
Soft skills
mentoringcollaborationcommunicationproblem-solvinginnovation
