FREE ACCESS
5,000–10,000 jobs/day
See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Machine Learning Engineer, Core Evaluations
CantinaMachine Learning Engineer focusing on audio model evaluation at Cantina Labs. Designing comprehensive model evaluation pipelines and leading evaluation efforts for the team.
About the role
Key responsibilities & impact- Designing model evaluation pipelines for models in development and production
- Designing user studies for subjective model evaluations.
- Converting requirements into measurable metrics.
- Designing and developing automated evaluation dashboard to see model performances and compare results.
- Training new models to capture new and different evaluation metrics.
- Communicating with the model team to help design better models based on the evaluation results.
- Communicating with the data team to help decide the type of data necessary to improve model performance.
- Communication with the product-manager to make sure product requirements are correctly measured.
- Help grow the evaluation team as the founding member.
- Lead the evaluation team in the future.
Requirements
What you’ll need- Strong experience and intuition for designing metrics that capture model performance.
- Strong experience with designing user studies on Mechanical Turk or similar platforms. .
- Strong experience with model training and fine-tuning for model evaluation.
- Strong statistical knowledge and experience to statistically compare evaluation results and take decisions.
- Very strong engineering and programming skills.
- Experience with training ASR, TTS models.
- Experience at ML teams working on large-scale machine learning problems. (>3B models with >1m hours of data)
Benefits
Comp & perks- Please Note: to ensure that candidates select the most relevant jobs for their skills, we have set up limits to the number of times candidates can apply. The following limits apply to all roles at Cantina:
- - Candidates may not apply more than 3 times in any 60 days for any job
- - Candidates may not re-apply to any role within 30 days if not presented with an offer (for re-applying to the same job, the limit is 90 days)