Executive Director, Functional AI, Process Excellence
Novartis
AI Data Quality Lead
TaskUs
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇺🇸 United States
Visit company websiteJob Level
Senior
Tech Stack
PythonSQL
About the role
- Drive the development, refinement, and documentation of quality assurance processes and standard operating procedures to ensure high-quality outputs
- Establish comprehensive quality metrics (e.g. F1 score, inter-annotator agreement) that align with business objectives and industry standards
- Continuously review and refine annotation workflows to proactively identify risks and areas to increase efficiency and reduce errors
- Act as the subject matter expert on annotation quality, providing ongoing feedback, training, and support to annotators and project teams to uphold the highest quality standards
- Lead in-depth data analysis to diagnose quality issues, assess the effectiveness of quality strategies, and uncover root causes of recurring errors
- Develop and maintain dashboards that provide real-time insights into quality metrics and project performance
- Prepare and deliver strategic quality reports to senior management and clients, articulating quality trends, risks, and improvement plans
- Partner with cross-functional teams, including operational management, engineering, and client services, to align on project goals and quality assurance initiatives
- Lead a team of Data Quality Analysts and provide mentorship, training, and expertise, fostering a culture of continuous improvement and accountability
- Manage the configuration and integration of annotation and quality control tools (e.g. Labelbox, Dataloop, LabelStudio), ensuring optimal tool performance and alignment with project requirements
- Identify, evaluate, and implement innovative quality control tools and automation technologies to streamline quality control workflows, enhance analytical capabilities, and improve operational efficiency.
Requirements
- Bachelor's degree in a technical field (e.g. Computer Science, Data Science) or equivalent professional experience
- 3+ years of experience in data quality management, data operations, or related roles within AI/ML or data annotation environments
- Proven track record in designing and executing quality assurance strategies for large-scale, multi-modal data annotation projects
- Proven track record in a leadership role managing and developing high-performing, remote or distributed teams
- Deep understanding of data annotation processes, quality assurance methodologies, and statistical quality metrics (e.g., F1 score, inter-annotator agreement)
- Strong data-analysis skills, with the ability to interrogate large datasets, perform statistical analyses, and translate findings into actionable recommendations
- Excellent communication skills, with experience presenting complex data and quality insights to technical and non-technical stakeholders
- Proficiency with annotation and QA tools (e.g., Labelbox, Dataloop, LabelStudio)
- High-level of proficiency in common data-analysis tools, such as Excel and Google Sheets
- Familiarity with programmatic data analysis techniques (e.g. Python, SQL)
- Familiarity with the core concepts of AI/ML pipelines, including data preparation, model training, and evaluation.
Benefits
- competitive industry salaries
- comprehensive benefits packages
- inclusive environment
- positive impact on the community
- professional growth at all stages of an employee's career
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data quality managementdata operationsquality assurance strategiesdata analysisstatistical analysisprogrammatic data analysisPythonSQLExcelGoogle Sheets
Soft skills
leadershipmentorshipcommunicationteam managementcontinuous improvementanalytical thinkingproblem-solvingcollaborationtrainingfeedback
