Salary
💰 $108,600 - $155,100 per year
Tech Stack
AirflowAWSCloudPandasPythonSparkSQL
About the role
- Be the primary owner of the data needs of a specific product vertical, from raw-data ingestion to end-user analysis
- Build intuitive, easy-to-use, and high quality datasets using Spark and DBT
- Build pipelines to assist the Data Science team in running product experiments (including DSA related experimentation pipelines)
- Track down data quality issues and set appropriate data quality monitors and alerts
- Participate in the occasional on-call rotation (12-hr day time shifts, ~1-2/month)
- Understand Data Science & experiment related datasets, document and educate team on table/field usage
- Serve as the liaison between your product vertical and the core data infrastructure team to meet business needs efficiently
- Support your vertical with data-related automation and mentor junior engineers
- Contribute to the strategic direction of data engineering across the company
- Build specialized data products that empower non-engineering teams with deeper insights into their datasets
Requirements
- BSc/MSc in Computer Science or related field
- 7+ years of industry experience, at least 5 of which are in a data engineering (or data-centric software engineering) role
- Track record of building and operating scalable, flexible, and always-on data pipelines focused on product experimentation and Data Science needs
- Fluent in data manipulation and SQL
- Strong knowledge of Spark
- Strong knowledge of Python (libraries like pandas and polars, helpful)
- Strong experience with Databricks (SQL Warehouses, Jobs Clusters, and Serverless)
- Familiarity with DBT and Airflow
- Experience with cloud providers and cloud storage (AWS preferred)
- Experience with efficiently working with datasets at TB scale
- Experience with StatSig product platform is preferred
- Service and data quality oriented
- A passion for shipping production quality code with good test coverage
- Ability to prioritize tasks and self-motivate without constant supervision
- Willingness to travel to Toronto office up to 2 days/week