Spotify

Senior Data Engineer

Spotify

full-time

Posted on:

Origin:  • 🇺🇸 United States • New York

Visit company website
✨ AI Apply
Manual Apply

Salary

💰 $160,091 - $228,702 per year

Job Level

Senior

Tech Stack

BigQueryGoogle Cloud PlatformJavaScala

About the role

  • Design, develop and maintain robust, scalable batch and streaming pipelines using Scio, GCP (BigQuery, Dataflow, GCS), and our orchestration stack including Flyte and Styx.
  • Build and optimize high-throughput, low-latency pipelines for exciting projects such as Contextual Targeting, Custom Website (ie Pixel-based) Audiences, Artist Affinity audiences and more.
  • Collaborate closely with Machine Learning and Staff engineers to design and build strategic products such as the ML-driven 1st party Audiences and the next generation of Lookalike Audiences.
  • Take charge of the architecture and development of upcoming key projects such as Pre and Post Campaign Insights, and the creation of Audiences based on the unique data we have about our users.
  • Champion scalability, testability, maintainability, reliability and high-quality code, ensuring we follow Spotify’s Golden Path technologies.
  • Work in a T-shaped environment, collaborating with engineers, product managers, and other stakeholders, embracing learning and leadership opportunities daily.
  • Lead on data architecture and design decisions when needed, while also being comfortable executing on plans, pair-programming and debugging with your teammates.
  • Proven experience working with distributed, high-volume production data pipelines and services, and heterogeneous data and tools such as BigQuery and BigTable
  • Strong programming skills in Java and/or Scala, with hands-on experience in frameworks like Scio
  • Commitment to agile software processes, data-driven development, reliability, and responsible experimentation
  • Hands-on experience with orchestration tools like Flyte and Styx
  • Excellent problem-solving and communication skills
  • Strong interest in, and ideally some experience with, either machine learning or backend development
  • We offer you the flexibility to work where you work best! For this role, you can be within the North American region as long as we have a work location.
  • This team operates within the EST time zone for collaboration.

Requirements

  • Proven experience working with distributed, high-volume production data pipelines and services, and heterogeneous data and tools such as BigQuery and BigTable
  • Strong programming skills in Java and/or Scala, with hands-on experience in frameworks like Scio
  • Commitment to agile software processes, data-driven development, reliability, and responsible experimentation
  • Hands-on experience with orchestration tools like Flyte and Styx
  • Excellent problem-solving and communication skills
  • Strong interest in, and ideally some experience with, either machine learning or backend development