Design and own production AI systems end-to-end (LLM pipelines, RAG, reranking, vector stores, orchestration)
Make thoughtful build/don’t-build decisions based on real data
Evolve our AI stack over time, from model infrastructure to workflow orchestration to evaluation tooling
Design evaluation systems that measure retrieval quality, reasoning accuracy, end-to-end performance
Build tooling that helps the team iterate confidently and catch regressions early
Define how we measure success across the platform
Analyze production outputs to identify failure patterns and root causes
Turn complex findings into clear technical recommendations
Know when to push forward and when to pause based on data
Be the go-to technical voice for AI architecture decisions
Influence standards for how LLM systems are built, tested, deployed, and monitored
Mentor senior engineers through design reviews and hands-on collaboration
Partner with product and compliance teams to translate domain complexity into clean technical solutions
Ship systems optimized for latency, cost, reliability, and auditability
Embed safety guardrails, confidence thresholds, and human-in-the-loop workflows
Ensure outputs are traceable and explainable

Requirements

10+ years of software engineering experience, including 3+ years working directly on ML/AI systems
Real ownership of production LLM systems
Deep experience with RAG, embeddings, reranking, vector databases (Pinecone, FAISS, Chroma, etc.), and agentic workflows
Experience designing evaluation frameworks and using quantitative analysis to improve system performance
Strong Python skills (TypeScript is a plus)
A track record of making architectural decisions that shape team direction
Experience operating AI systems in production - observability, reliability, cost tradeoffs
The ability to break down ambiguous, high-stakes problems into structured investigations
Clear communication skills and comfort working cross-functionally

Benefits

Up to 100% employer-paid premiums for medical, dental, and vision coverage for employees and their dependents
Comprehensive wellness benefits and healthcare concierge services
401(k) plan with company-paid life and disability insurance
Tax-advantaged spending accounts and discounted voluntary offerings
Paid Parental Leave policy after six months of employment
Access to Kindbody fertility and family-building benefits
Generous annual stipends for professional and personal development
Flexible vacation policy and paid holidays

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

AI systemsLLM pipelinesRAGrerankingvector storesevaluation frameworksPythonTypeScriptquantitative analysisarchitectural decisions

Soft Skills

clear communicationmentoringcross-functional collaborationproblem-solvinginfluencing standardstechnical recommendationsdata-driven decision makingteam leadershipiterative developmentanalytical thinking