Staff AI Scientist

Fiddler AI

Lead applied AI research and development at Fiddler, focusing on trust and transparency in AI applications. Collaborate with engineering and product teams to enhance customer AI observability.

Posted 5/7/2026full-timePalo Alto • California • 🇺🇸 United StatesLead💰 $190,000 - $300,000 per yearWebsite

Tech Stack

Tools & technologies

PythonPyTorch

About the role

Key responsibilities & impact

Lead applied research and development for the models and datasets at the core of Fiddler's Trust Service and suite of guardrail classifiers and evaluators that customers depend on to keep their LLM and agentic applications safe, accurate, and compliant in production.
Partner closely with other engineering teams, Product, and Customer Success to build strong relationships with customer data science and ML engineering teams.
Design, train, and ship production classifiers for safety, security, and quality detection under strict latency and cost constraints.
Lead the development of synthetic and adversarial dataset pipelines, including novel methods for generating, filtering, and validating data that exposes failure modes our models need to learn.
Drive the technical direction of generative insights – the LLM- and agent-powered analysis layer that helps customers diagnose what's going wrong in their AI applications and why.
Contribute to the evaluation and experimentation infrastructure that lets the AI Science team and our customers reliably measure model quality, regression, and drift across rapidly evolving model populations.
Explore reinforcement learning and preference-based methods where they offer real leverage over supervised baselines.
Collaborate with Backend and Platform engineers to take research prototypes from notebook to a hardened, scaled, observable service.
Partner with Product, Solutions Engineering, and Customer Success to translate enterprise customer needs into research problems and translate research results back into product.
Mentor AI Scientists on the team, raise the technical bar through code review and design review, and represent Fiddler externally through publications, talks, or open-source contributions when appropriate.

Requirements

What you’ll need

7+ years of applied AI experience, with a strong track record of taking models from research to production
Experience in LLM or Agentic Evals, Guardrailing
Deep expertise training and fine-tuning classifier models, including modern encoder architectures (BERT-family, ModernBERT, etc.) and LLM-as-classifier approaches; clear understanding of the tradeoffs between them
Hands-on experience with dataset development as a first-class engineering discipline: sourcing, labeling, synthetic generation, adversarial augmentation, and quality control
Strong applied experience with LLMs and agentic systems – prompting, fine-tuning, and evaluation
Proficiency in Python and the modern ML stack (PyTorch, Hugging Face, common training/serving frameworks)
Comfortable working in production environments and partnering with backend and platform engineers on real-time inference, monitoring, and rollout
Excited by the prospect of using AI coding tools in your own workflow to push the limits of what one engineer can ship – responsibly, with a clear eye on quality, security, and the failure modes these tools introduce
Excellent written and verbal communication; able to explain research tradeoffs to engineers, PMs, and customers.

Benefits

Comp & perks

Unlimited PTO
Competitive pay + equity
Premium health, dental & vision (100% premiums covered for employee)
401(k) plan
Monthly fitness reimbursement
Paid parental leave
Free annual Caltrain pass
Monthly in-office massages
Fastrak reimbursement
Free lunch Mon–Thurs

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

applied AIclassifier modelsdataset developmentsynthetic generationadversarial augmentationLLM-as-classifierPythonPyTorchHugging Facemodern encoder architectures

Soft Skills

communicationmentoringcollaborationrelationship buildingtechnical directioncode reviewdesign review