Machine Learning Engineer, Core Evaluations

Cantina

Machine Learning Engineer focusing on audio model evaluation at social AI company Cantina Labs. Designing evaluation pipelines and automating reporting for speech generation and recognition models.

Posted 4/29/2026full-timeRemote • 🇺🇸 United StatesMid-LevelSeniorWebsite

About the role

Key responsibilities & impact

Designing model evaluation pipelines for models in development and production
Designing user studies for subjective model evaluations.
Converting requirements into measurable metrics.
Designing and developing automated evaluation dashboard to see model performances and compare results.
Training new models to capture new and different evaluation metrics.
Communicating with the model team to help design better models based on the evaluation results.
Communicating with the data team to help decide the type of data necessary to improve model performance.
Communication with the product-manager to make sure product requirements are correctly measured.
Help grow the evaluation team as the founding member.
Lead the evaluation team in the future.

Requirements

What you’ll need

Strong experience and intuition for designing metrics that capture model performance.
Strong experience with designing user studies on Mechanical Turk or similar platforms.
Strong experience with model training and fine-tuning for model evaluation.
Strong statistical knowledge and experience to statistically compare evaluation results and take decisions.
Very strong engineering and programming skills.
Experience with training ASR, TTS models.
Experience at ML teams working on large-scale machine learning problems. (>3B models with >1m hours of data)

Benefits

Comp & perks

📊 Check your resume score for this job Improve your chances of getting an interview by checking your resume score before you apply. Check Resume Score

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

model evaluationuser studies designmetrics designautomated evaluation dashboardmodel trainingfine-tuningstatistical analysisASR modelsTTS modelslarge-scale machine learning

Soft Skills

communicationteam leadershipcollaborationintuition for metrics design