Scientific Content Lead
Arena
full-time
Posted on:
Location Type: Hybrid
Location: Bay Area • California • United States
Visit company websiteExplore more
Job Level
About the role
- Own Arena’s scientific communications strategy, ensuring that our evaluation methodology, benchmarks, and data quality practices are clearly understood and accurately represented externally.
- Lead Arena’s proactive data quality narrative, defending against common critiques and mischaracterizations through transparency, evidence, and high-integrity storytelling.
- Develop canonical explanations of Arena’s measurement approach, including Bradley-Terry-Luce-style ranking, confidence intervals, and uncertainty-aware interpretation.
- Ensure that Arena’s leaderboards are communicated responsibly: rankings are statistical estimates, small differences are often noise, and uncertainty must be preserved in public interpretation.
- Anticipate, track, and respond to methodological critiques, especially around contamination, overfitting, gaming, distribution shift, and evaluation validity.
- Partner closely with researchers to translate technical work into rigorous public materials, including methodology documentation, research posts, and open-source releases.
- Support Arena’s Academic Partnerships Program, strengthening scientific connectivity through collaborations, citations, and peer-reviewed credibility.
- Create briefing materials for high-stakes audiences, including frontier AI labs, policymakers, analysts, and enterprise partners, ensuring that technical nuance survives external scrutiny.
- Serve as a scientific editor and reviewer across external communications, stress-testing claims before they become public narratives.
Requirements
- 8-10 years of experience in AI/ML, evaluation, research, or scientific communications, with deep familiarity in how frontier model performance is measured and debated.
- Strong technical background in machine learning, benchmarking, or model evaluation, with the credibility to engage directly with leading labs and researchers.
- Exceptional writing and communication skills, especially the ability to explain complex methodology clearly without oversimplifying or overstating conclusions.
- Track record of producing scientifically rigorous external-facing work, such as technical publications, evaluation reports, methodology documentation, or research translation.
- Deep comfort operating in ambiguity, where uncertainty, tradeoffs, and limitations must be communicated transparently rather than smoothed over.
- High editorial judgment and the ability to identify where scientific nuance is most likely to be misunderstood or weaponized.
- Collaborative mindset and experience partnering across research, product, policy, and communications teams.
Benefits
- Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs.
- Competitive compensation and equity aligned to the markets where our team members are based.
- The opportunity to work on cutting-edge AI with a small, mission-driven team.
- A culture that values transparency, trust, and community impact.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
machine learningbenchmarkingmodel evaluationBradley-Terry-Luce-style rankingconfidence intervalsuncertainty-aware interpretationscientific communicationsevaluation methodologydata quality practicesresearch translation
Soft Skills
exceptional writingcommunication skillshigh editorial judgmentcollaborative mindsetability to operate in ambiguitytransparencyscientific storytellingcritical thinkingattention to detailability to engage with researchers