Forward Deployed Engineer
Take2 AI
full-time
Posted on:
Location Type: Hybrid
Location: New York City • New York • United States
Visit company websiteExplore more
Salary
💰 $80,000 - $140,000 per year
About the role
- Lead technical onboarding with customers to understand roles, hiring goals, must-have signals, and constraints.
- Translate customer needs into structured interview flows, role-specific question banks, and scoring rubrics.
- Set clear expectations on what “good” looks like (pass/fail thresholds, evaluation rationale, interviewer tone and style).
- Design, build, and refine prompts and agent logic that drive interviewer behavior, question sequencing, probing, and candidate experience.
- Ensure interviewer conversations are consistent, role-relevant, and robust to edge cases (evasive candidates, unclear answers, noisy audio, interruptions).
- Implement multi-step structured interview flows with state management and guardrails.
- Design and maintain AI-based evaluation and scoring aligned to customer rubrics and hiring criteria.
- Improve accuracy, consistency, and explainability of scoring at scale (including calibration across roles/customers).
- Identify bias/fairness risks and contribute to mitigation strategies and compliant evaluation practices.
- Launch new customer interviewers into production and own iteration cycles from early rollout through steady-state performance.
- Use customer feedback + production metrics to prioritize improvements and deliver measurable outcomes.
- Communicate changes clearly to customers and internal stakeholders.
- Build and own lightweight QA/evaluation pipelines to measure conversation quality, scoring accuracy, and reliability before/after changes.
- Monitor production performance and partner with engineering to balance quality, latency, and cost tradeoffs.
- Contribute to standards and best practices for prompt quality, eval quality, and voice-agent reliability.
Requirements
- 2+ years working with LLMs, NLP systems, or AI agents in production.
- Demonstrated experience designing and deploying agent workflows (prompts + structured flows) that operate at scale.
- Strong understanding of prompt engineering, agent control, failure modes, and conversational edge cases.
- Experience building or contributing to evaluation/testing/QA frameworks for AI systems.
- Comfort being customer-facing: running technical discovery, translating requirements, and driving onboarding to production.
- Strong analytical mindset (accuracy, consistency, bias, calibration, and edge cases).
Benefits
- 📊 Check your resume score for this job Improve your chances of getting an interview by checking your resume score before you apply. Check Resume Score
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
LLMsNLP systemsAI agentsprompt engineeringagent workflowsevaluation frameworksQA frameworksstructured interview flowsscoring rubricsstate management
Soft Skills
customer-facinganalytical mindsetcommunicationcollaborationproblem-solvingadaptabilityattention to detailstakeholder managementfeedback incorporationiteration management