Codvo.ai

QA Auditor, Sampling, Quality Gates, Release Readiness

Codvo.ai

full-time

Posted on:

Location Type: Remote

Location: United States

Visit company website

Explore more

✨ AI Apply
Apply

About the role

  • Protect dataset quality at scale by running sampling-based audits, enforcing quality gates, and ensuring every dataset release is audit-ready and meets acceptance thresholds.
  • Execute sampling plans (stratified by label type, difficulty, site/surgeon/device, model confidence) for ongoing QA and release audits.
  • Audit annotations against guidelines + ontology; score defects using standardized rubrics (severity, type, root cause).
  • Run and validate quality gates prior to release: completeness/coverage checks schema/constraint checks (illegal combinations, temporal consistency, boundary rules) defect density and rework thresholds.
  • Generate rework tickets with precise instructions; track closure and verify fixes.
  • Partner with Annotation Lead to drive calibration actions (training refreshers, guideline clarifications, examples library).
  • Maintain QA dashboards: defect rate, rework %, gate pass rate, audit cycle time, top error patterns by annotator/team/label type.
  • Support release readiness: ensure required artifacts are complete for the Evidence Pack (QA results, sampling logs, sign-offs).
  • Escalate systematic issues to Adjudication/Clinical Review (definition gaps, ambiguous classes, recurring edge cases).

Requirements

  • 3–7+ years in data QA / annotation QA / operations quality; experience auditing labeled datasets (video preferred).
  • Strong command of sampling-based QA: stratified sampling, acceptance thresholds, and audit scoring.
  • High attention to detail; comfortable reading and enforcing detailed SOPs/guidelines.
  • Strong documentation skills; can write clear defect descriptions and rework instructions.
  • Comfortable with dashboards/spreadsheets; basic metric literacy (defect density, rework, throughput).
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
sampling-based auditsquality gatesaudit readinessdefect scoringschema checksdefect densityrework thresholdscalibration actionsQA dashboardsmetric literacy
Soft Skills
attention to detaildocumentation skillsclear communication