AI Data Quality Analyst
TaskUs
full-time
Posted on:
Location Type: Remote
Location: Philippines
Visit company websiteExplore more
Job Level
Tech Stack
About the role
- Perform quality audits on annotated datasets to ensure that they meet established guidelines and quality benchmarks.
- Leverage statistical based quality metrics such as F1 score and inter-annotator agreement to evaluate data quality.
- Analyze annotation errors, trends, project processes, and project documentation to identify and understand the root cause of errors and propose remediation strategies.
- Resolve and analyze edge-case annotations to ensure quality and identify areas for improvement.
- Become proficient in using annotation and quality control tools to perform reviews and track quality metrics.
- Become an expert in the project specific guidelines and provide feedback for potential clarifications or improvements.
- Identify opportunities to use automation to help enhance analytics, provide deeper insights, and improve efficiency.
- Develop and maintain up-to-date documentation on quality standards, annotation guidelines, and quality control procedures.
- Provide regular feedback that identifies areas for improvement across the annotation pipeline.
- Work closely with key project stakeholders and clients to understand project requirements and improve annotation pipelines.
- Assist with training annotators, providing guidance, feedback, and support to ensure data quality.
- Provide regular updates that highlight data quality metrics, key findings, and actionable insights for continuous process improvements.
Requirements
- 1+ years of experience as a data analyst with exposure to data quality and/or data annotation - ideally within an AI/ML context.
- Familiarity with the basic concepts of AI/ML pipelines and data.
- Strong analytical and problem-solving skills with an exceptional eye for detail.
- Excellent written and verbal communication skills, with the ability to clearly articulate quality issues and collaborate with diverse teams.
- Ability to work independently and manage time effectively to meet deadlines.
- A strong problem-solver who thinks critically and drives innovation and continuous optimization.
- A quick learner with the ability to work independently in a fast-paced environment.
- A strong focus on detail, balanced against strategic priorities.
- A positive can-do attitude and the ability to easily adapt to new environments.
- Not afraid to speak up.
- Familiarity with data annotation tools (e.g. Labelbox, Dataloop, LabelStudio etc.) is a nice to have.
- Experience working with multi-modal AI/ML datasets (images, videos, text, audio) is a nice to have.
- Prior experience in an agile or fast-paced tech environment with exposure to AI/ML pipelines is a nice to have.
- Knowledge of programming languages (e.g. Python) is a nice to have.
- Knowledge of the concepts and principles of data quality for AI/ML models and the impacts it can have on model performance is a nice to have.
- Working understanding of common quality metrics and statistical methods used in AI/ML data quality is a nice to have.
- Knowledge of AI/ML concepts and experience with data for AI/ML models is a nice to have.
- Experience in prompt engineering and leveraging LLMs in your day-to-day work is a nice to have.
Benefits
- Competitive industry salaries
- Comprehensive benefits packages
- Wellness programs
- Professional development opportunities
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
data qualitydata annotationstatistical methodsF1 scoreinter-annotator agreementdata analysisprogramming languagesAI/ML conceptsprompt engineeringmulti-modal datasets
Soft skills
analytical skillsproblem-solving skillsattention to detailwritten communicationverbal communicationtime managementcritical thinkingadaptabilitycollaborationinnovation