AI Applied Scientist

Wizard

Applied Scientist measuring and improving the accuracy of AI shopping agent at Wizard. Collaborating with ML and AI Engineering to enhance evaluation frameworks and metrics.

Posted 5/21/2026full-timeRemote • 🇺🇸 United StatesMid-LevelSenior💰 $225,000 - $280,000 per yearWebsite

About the role

Key responsibilities & impact

Define and evolve accuracy metrics across the full shopping experience (retrieval, ranking, recommendations, outcomes)
Design and run experiments to measure improvements and regressions
Build and maintain evaluation datasets, benchmarks, and scoring frameworks
Improve the LLM judges that power our evaluation pipeline: prompting, calibration, and fine-tuning where it matters
Translate ambiguous product questions into clear, measurable hypotheses and analysis
Partner with ML Engineers to validate model changes and guide iteration
Identify failure modes and edge cases, and drive improvements through data
Make agent performance visible, trusted, and actionable across product and engineering

Requirements

What you’ll need

5+ years in Applied ML, AI Research, or Applied Science (PhD or equivalent depth strongly preferred)
Hands-on experience evaluating modern AI/ML systems: LLMs, agents, ranking, or recommendations
Direct experience with LLM-based systems: judge models, RAG, prompt engineering, fine-tuning, RLHF, or similar
Strong experimentation foundations: A/B testing, causal inference, statistical rigor
Proven ability to operate in ambiguity: defining problems, not just solving pre-defined ones
Clear, structured communication that influences across ML, engineering, and product.

Benefits

Comp & perks

Equity in the form of stock options
Medical, dental, and vision coverage
401(k) plan
Flexible PTO and company holidays
Fully remote work within the United States
Periodic company offsites and team gatherings

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Applied MLAI ResearchLLMsrankingrecommendationsprompt engineeringfine-tuningA/B testingcausal inferencestatistical rigor

Soft Skills

problem definitionstructured communicationinfluencingoperating in ambiguity

Certifications

PhD