Machine Learning Scientist – Open Source Lead

Arena

Machine Learning Scientist leading open-source research at LMArena for evaluating AI models. Designing experiments and curating datasets to advance understanding of AI model performance.

Posted 4/12/2026full-timeBay Area • California • 🇺🇸 United StatesSeniorWebsite

Tech Stack

Tools & technologies

PythonPyTorchTensorflow

About the role

Key responsibilities & impact

LMArena is looking for a Machine Learning Scientist to lead our open-source research.
You'll design, run, and share new methods and experiments that reveal what makes models useful, trustworthy, and capable.
You’ll be responsible for curating high-impact datasets, developing new methodology and reproducible benchmarks, and releasing code.
Collaborate with engineers to implement and scale research findings into production systems.
Analyze large-scale human voting and interaction data to uncover insights into model performance and user preferences.

Requirements

What you’ll need

PhD or equivalent research experience in Machine Learning, Natural Language Processing, Statistics, or a related field.
Strong understanding of LLMs and modern deep learning architectures (e.g., Transformers, diffusion models, reinforcement learning with human feedback).
Proficiency in Python and ML research libraries such as PyTorch, JAX, or TensorFlow.
Demonstrated ability to design and analyze experiments with statistical rigor.
Experience publishing research or working on open-source projects in ML, NLP, or AI evaluation.
Comfortable working with real-world usage data and designing metrics beyond standard benchmarks.
Ability to translate research questions into practical systems and collaborate across engineering and product teams.
Passion for open science, reproducibility, and community-driven research.

Benefits

Comp & perks

Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs.
The opportunity to work on cutting-edge AI with a small, mission-driven team.
A culture that values transparency, trust, and community impact.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Machine LearningNatural Language ProcessingStatisticsLLMsdeep learning architecturesTransformersdiffusion modelsreinforcement learningPythonML research libraries

Soft Skills

collaborationdesigning experimentsstatistical rigortranslating research questionscommunity-driven researchpassion for open science

Certifications

PhD