Annotation QA Analyst – Content Platform

Spotify

Annotation QA Analyst at Spotify responsible for managing annotated data quality for ML models. Collaborating with Product and Engineering teams to enrich consumer experiences across music, podcasts, and audiobooks.

Posted 4/28/2026full-timeNew York City • New York • 🇺🇸 United StatesMid-LevelSenior💰 $82,921 - $118,459 per yearWebsite

Tech Stack

Tools & technologies

SQL

About the role

Key responsibilities & impact

Review annotated data to ensure it meets Spotify’s quality standards and policies, prioritizing work based on business needs
Deliver high-quality, timely results for Product and Engineering teams using established QA frameworks and metrics such as agreement rates and consensus
Handle complex edge cases, helping define ground truth and reduce ambiguity across datasets
Identify patterns, insights, and areas for improvement, and communicate findings clearly to both technical and non-technical partners
Contribute to feedback loops between annotation teams, R&D collaborators, and content policy experts to improve workflows and outputs
Help develop and refine annotation guidelines, supporting annotator training and continuous improvement
Collaborate closely with teammates across multiple projects and domains

Requirements

What you’ll need

experience working with annotation, data quality, or QA processes in ML/AI environments
familiar with LLM or AI-driven annotation workflows and human-in-the-loop systems
comfortable reviewing large-scale datasets across different modalities such as text, audio, images, or video
care about quality and consistency, with a structured approach to evaluating data
communicate clearly and can explain complex ideas in a simple, accessible way
collaborate well with cross-functional partners in fast-moving environments
solid understanding of the machine learning lifecycle, from data collection to deployment
comfortable working with emerging AI tools and agent workflows
curiosity and interest in music, podcasts, or audiobooks
familiarity with SQL or music metadata standards (nice to have)

Benefits

Comp & perks

health insurance
six month paid parental leave
401(k) retirement plan
monthly meal allowance
23 paid days off
13 paid flexible holidays

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

annotationdata qualityQA processesmachine learning lifecycleSQLAI-driven annotation workflowshuman-in-the-loop systemsdata evaluationlarge-scale datasetsmusic metadata standards

Soft Skills

communicationcollaborationcuriositystructured approachproblem-solvingadaptabilityclear explanation of complex ideasfeedback integrationcross-functional teamworkattention to detail