Drata

Senior Applied AI Engineer

Drata

full-time

Posted on:

Location Type: Hybrid

Location: San FranciscoCaliforniaUnited States

Visit company website

Explore more

AI Apply
Apply

Salary

💰 $192,000 - $237,100 per year

Job Level

Tech Stack

About the role

  • Design and evaluate information access + reasoning strategies across RAG, agents, and classic ML: chunking, embedding models, hybrid search, metadata filtering, structured retrieval, tool use, and multi-step workflows
  • Prototype GenAI workflows (including agentic systems) that map and reason over compliance objects (controls ↔ risks ↔ requirements)
  • Explore ML + probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and weak supervision
  • Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection
  • Implement and tune ranking/reranking systems: cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring functions
  • Run experiments to validate hypotheses and quantify improvements before production rollout
  • Debug failure modes and build error taxonomies across retrieval, reasoning, and generation
  • Collaborate with AI and Software Engineers to hand off validated approaches for productionization
  • Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product

Requirements

  • 5+ years of experience in applied research, data science, or ML engineering with focus on NLP or information retrieval
  • Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance
  • Experience with RAG systems: chunking strategies, vector databases, retrieval optimization
  • Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical analysis
  • Strong Python skills and comfort with notebook-driven research workflows
  • Experience communicating research findings to engineering teams and translating insights into actionable recommendations
  • Bonus: Experience with compliance, legal, or document-heavy domains
  • Bonus: Publications or contributions in IR, NLP, or RAG evaluation
Benefits
  • Up to 100% employer-paid premiums for medical, dental, and vision coverage for employees and their dependents
  • Comprehensive wellness benefits and healthcare concierge services
  • 401(k) plan
  • Company-paid life and disability insurance
  • Tax-advantaged spending accounts
  • Paid Parental Leave policy
  • Kindbody fertility and family-building benefits
  • Generous annual stipends for both professional and personal development
  • Flexible vacation policy, paid holidays, and other perks
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
information retrievalembedding modelschunking strategiesvector databasesranking modelsevaluation methodologyA/B testingPythonstatistical analysisGenAI workflows
Soft Skills
communicationcollaborationtranslating insightsdebuggingprototyping