Design, implement, and deploy robust data solutions for both batch ingestion and stream processing using Scala and Python with frameworks like Spark, primarily deployed via Databricks and Airflow.
Establish and enforce rigorous standards for code development, testing, and deployment to guarantee the highest levels of data quality, accuracy, and governance.
Extend functionality of Product Performance Data offerings by evolving existing data models and developing new data pipelines to introduce novel metrics and dimensions critical for measuring Disney+ engagement.
Maintain, update, and expand existing source code that powers core datasets, ensuring strict quality standards and high data reliability.
Drive platform and business success through active collaboration with Product Managers, Data Analysts, and cross-functional engineering teams to define requirements and deliver new data features.
Work within a modern data and engineering tech stack, including Airflow, Spark, Databricks, Delta Lake, Snowflake, GitHub Actions, and Jenkins.
Proactively investigate and resolve complex data pipeline failures and quality issues, performing root cause analysis, and creating comprehensive documentation to ensure data integrity and support operational efficiency.
Identify and implement technical optimizations to improve performance, scalability, and cost-efficiency of existing data pipelines, while providing technical guidance and mentorship on best practices.

Requirements

5+ years of software engineering experience developing large data pipelines
Strong CICD and SDLC fundamentals
Strong algorithmic problem-solving expertise
Strong fundamental Scala and Python software engineering skills
Basic understanding of AWS or other cloud provider resources (S3)
Strong SQL skills and ability to create queries to analyze complex datasets
Hands-on production environment experience with distributed engineering systems such as Spark and Kafka
Hands-on production experience with orchestration systems such as Airflow
Some scripting language (Bash, PowerShell) experience
Willingness and ability to learn and pick up new skillsets
Self-starting problem solver with an eye for detail and excellent analytical and communication skills.

Benefits

A bonus and/or long-term incentive units may be provided as part of the compensation package, in addition to the full range of medical, financial, and/or other benefits, dependent on the level and position offered.

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

ScalaPythonSparkSQLCICDSDLCdata pipelinesalgorithmic problem-solvingBashPowerShell

Soft Skills

problem solveranalytical skillscommunication skillsattention to detailself-startingcollaborationmentorshiptechnical guidance