Machine Learning Engineer – Model Evaluation, Experimentation

Mercor

Machine learning engineer authoring experiments and evaluating frontier AI models for a leading AI lab. Implementing ML and reinforcement-learning tasks, analyzing results, and identifying model limitations remotely in the United States.

Posted 8/9/2026full-timeRemote • 🇺🇸 United StatesJunior💰 $60 - $90 per hourWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and implementing machine learning tasks, with a strong focus on reinforcement learning concepts and model evaluation. Proficient in Python and Git, with a solid foundation in research methodologies and experimental analysis.

Highest-signal resume keywords

MSc Or PhD In Machine LearningHands-On Experience Training ML ModelsStrong Familiarity With Large Language ModelsBasic Understanding Of Reinforcement LearningStrong Written Communication Skills

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Machine LearningModel EvaluationExperiment AnalysisReinforcement LearningTask DesignPythonGitScriptingNotebook EnvironmentsAI Training

Soft Skills

Attention To DetailCreativityIndependent Problem Solving

Industry Keywords

Research EngineeringTraining BehaviorReward FunctionsEvaluation TechniquesResearch Methodologies

Tech Stack

Tools & technologies

Python

About the role

Key responsibilities & impact

Design well-defined, multi-step machine-learning tasks from real ML research ideas
Implement changes, run training experiments, and analyze results to establish correct solutions
Build tasks around reinforcement-learning concepts such as reward functions and training behavior
Evaluate frontier models and identify where and why they fall short
Compare notes with researchers and fellow experts to maintain consistency, rigor, and fairness
Work in a tight feedback loop with the lab’s researchers

Requirements

What you’ll need

MSc or PhD in machine learning, computer science, or another STEM field, or equivalent practical experience in a research-heavy domain
1+ years of experience in a research or research-engineering role
Hands-on experience training and evaluating ML models and running experiments end-to-end, including setup, execution, and analysis
Strong familiarity with large language models, including their capabilities, limitations, and evaluation techniques
Working proficiency in Python and Git
Comfort working in scripting and notebook environments
Basic understanding of reinforcement learning, including reward functions and policy training, preferred
Past experience in AI training, model evaluation, or benchmark/task authoring preferred
High attention to detail and creativity in task design
Strong written communication skills
Ability to work independently through ambiguous, open-ended problems
Ability to engage reliably for approximately 35 hours per week

Benefits

Comp & perks

W-2 employment
Payroll, benefits, and compliance administered by Cincinnatus LLC
Fully remote work within the United States
Approximately 35 hours per week
Opportunity to be placed at a leading AI lab as part of its extended workforce
Reasonable accommodations for qualified individuals with disabilities and disabled veterans throughout the job application process