
Senior Applied AI Engineer
Drata
full-time
Posted on:
Location Type: Hybrid
Location: San Francisco • California • United States
Visit company websiteExplore more
Salary
💰 $192,000 - $237,100 per year
Job Level
Tech Stack
About the role
- Design and evaluate information access + reasoning strategies across RAG, agents, and classic ML: chunking, embedding models, hybrid search, metadata filtering, structured retrieval, tool use, and multi-step workflows
- Prototype GenAI workflows (including agentic systems) that map and reason over compliance objects (controls ↔ risks ↔ requirements)
- Explore ML + probabilistic approaches where GenAI is not the best fit: classifiers, ranking models, graph/link prediction, calibration, and weak supervision
- Build and maintain evaluation frameworks: golden datasets, automated quality metrics, regression detection
- Implement and tune ranking/reranking systems: cross-encoders, LLM-based rerankers, learning-to-rank, custom scoring functions
- Run experiments to validate hypotheses and quantify improvements before production rollout
- Debug failure modes and build error taxonomies across retrieval, reasoning, and generation
- Collaborate with AI and Software Engineers to hand off validated approaches for productionization
- Stay current on applied research in RAG, agents, LLM evaluation, and relevance modeling; bring innovations into the product
Requirements
- 5+ years of experience in applied research, data science, or ML engineering with focus on NLP or information retrieval
- Strong foundation in information retrieval: dense and sparse retrieval, embedding models, search relevance
- Experience with RAG systems: chunking strategies, vector databases, retrieval optimization
- Proficiency in evaluation methodology: metrics design, golden dataset creation, A/B testing, statistical analysis
- Strong Python skills and comfort with notebook-driven research workflows
- Experience communicating research findings to engineering teams and translating insights into actionable recommendations
- Bonus: Experience with compliance, legal, or document-heavy domains
- Bonus: Publications or contributions in IR, NLP, or RAG evaluation
Benefits
- Up to 100% employer-paid premiums for medical, dental, and vision coverage for employees and their dependents
- Comprehensive wellness benefits and healthcare concierge services
- 401(k) plan
- Company-paid life and disability insurance
- Tax-advantaged spending accounts
- Paid Parental Leave policy
- Kindbody fertility and family-building benefits
- Generous annual stipends for both professional and personal development
- Flexible vacation policy, paid holidays, and other perks
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
information retrievalembedding modelschunking strategiesvector databasesranking modelsevaluation methodologyA/B testingPythonstatistical analysisGenAI workflows
Soft Skills
communicationcollaborationtranslating insightsdebuggingprototyping