Data Scientist
Arena
full-time
Posted on:
Location Type: Hybrid
Location: Bay Area • California • United States
Visit company websiteExplore more
About the role
- Explore and analyze large, complex datasets to uncover patterns, biases, and causal relationships in model behavior and system performance.
- Formulate hypotheses about data quality, evaluation outcomes, and model performance — then design experiments to validate or refute them.
- Build reproducible analysis pipelines using Python, Pandas, NumPy, and Spark to process and interrogate large-scale data.
- Partner with ML researchers and engineers to design metrics and analyses that evaluate how models perform across domains, prompts, and tasks.
- Develop causal reasoning frameworks and statistical methods that help explain why models behave as they do — not just how well they perform.
- Communicate insights (for example, via blog posts) clearly to technical and non-technical partners, informing both research direction and infrastructure improvements.
Requirements
- 6+ years of experience in data science, ML analytics, or applied research, preferably in AI, ML, or large-scale data environments.
- Strong proficiency in Python, with deep experience in Pandas, NumPy, and distributed frameworks like Spark.
- Expertise in statistical modeling, causal inference, and experimental design.
- Experience reasoning about data distributions, sample quality, and the effects of data distribution shifts.
- Strong communication skills and the ability to collaborate closely with ML researchers and engineers.
- (Bonus) Background in AI model evaluation.
- (Bonus) Experience working with LLM outputs (for example, LLM-as-a-judge), embeddings, or other large-scale model artifacts.
- (Bonus) Experience with A/B testing.
Benefits
- Comprehensive health and wellness benefits, including medical, dental, vision, and additional support programs.
- The opportunity to work on cutting-edge AI with a small, mission-driven team
- A culture that values transparency, trust, and community impact
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data sciencemachine learning analyticsapplied researchstatistical modelingcausal inferenceexperimental designdata distributionsdata qualityA/B testinglarge-scale data processing
Soft Skills
strong communication skillscollaborationinsight communication