Design, develop, and maintain scalable ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
Optimize data workflows and SQL/Spark queries for performance, cost efficiency, and reliability.
Contribute to the design of our enterprise Data Lake and curated data assets.
Build robust ingestion frameworks, transformation logic, and reusable data engineering components.
Support data migrations and modernization efforts from legacy systems to cloud-based platforms.
Use AI coding copilots (Databricks Assistant, GitHub Copilot, SQL copilots) to accelerate development, code review, test creation, and documentation.
Automate repetitive tasks (schema evolution, config generation, logging templates, test data creation) with generative AI tools.
Leverage AI to auto-detect pipeline anomalies, data drift, quality issues, and performance bottlenecks.
Incorporate AI-assisted query optimization tools to improve Spark & SQL performance.
Work cross-functionally with Product, data consumers to deliver high-quality data solutions.
Participate in Agile ceremonies & contribute to sprint planning & technical design discussions.
Mentor junior data engineers, promote best practices in AI-assisted data engineering.
Ensure strong documentation, standards adoption, & continuous improvement across the team.

Requirements

5–7+ years of hands-on data engineering experience.
Advanced proficiency in Python, SQL, PySpark, and ETL/ELT frameworks.
Strong experience with Databricks and distributed data processing.
Demonstrated ability to optimize SQL/Spark workloads for performance and cost.
Experience using AI development tools such as: GitHub Copilot, Databricks Assistant, SQL/IDE AI copilots, AI-based data quality or monitoring tools
Solid understanding of data modeling, governance, and CI/CD for data pipelines.
Excellent communication skills and ability to work in a cross-functional environment.
Experience working in Agile environments with tools like Jira and Confluence.

Benefits

Medical, Vision and Dental benefits for you and your family, including Flexible Spending Accounts (FSA) and Health Savings Accounts (HSAs)
Generous paid time off policy including paid holidays, sick time and paid days off for your birthday
Free concert tickets
401(k) program with company match
Stock Program
New parent programs & support including caregiver leave and childcare cash, infertility support
Tuition reimbursement, student loan repayment internal growth and development programs & trainings
Volunteer time off, crowdfunding network

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

ETLELTPythonSQLPySparkDatabricksdata modelingCI/CDdata governancedata engineering

communicationcross-functional collaborationmentoringbest practices promotioncontinuous improvement