Lead Cloud Data Engineer – Manager
State Street
full-time
Posted on:
Location Type: Hybrid
Location: Hyderabad • India
Visit company websiteExplore more
Job Level
About the role
- Design, develop, and optimize end‑to‑end data pipelines using Databricks
- Implement and manage Lakehouse architecture with Delta tables
- Develop high‑performance PySpark and SQL transformations
- Ensure data quality, integrity, and performance across pipelines
- Lead troubleshooting, root cause analysis, and performance tuning
- Collaborate with Architects, Business Analysts, and downstream consumers
- Support CI/CD implementation for Databricks jobs and notebooks
- Mentor junior engineers and review code for quality and best practices
- Ensure adherence to security, governance, and compliance standards
Requirements
- 10-14 years of hands on experience with AWS Cloud Data Engineering with Databricks platform
- Expertise in Data Modeling and DB design
- Hands on experience in setting up data pipelines for ETL jobs across multiple data sources.
- Hands‑on experience with performance tuning, Git, Jenkins, and enterprise job scheduling tools (Autosys / Airflow)
- Basic understanding of Data warehousing and ETL concepts
- Knowledge of CI/CD tools (GitHub, Azure DevOps)
- Advanced SQL and Python skills
- Experience working in Agile delivery environments
Benefits
- inclusive development opportunities
- flexible work-life support
- paid volunteer days
- vibrant employee networks
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
DatabricksLakehouse architectureDelta tablesPySparkSQLETLData ModelingDB designperformance tuningAWS Cloud Data Engineering
Soft Skills
troubleshootingroot cause analysismentoringcollaborationcode review