Prompt Engineer
Take2 AI
full-time
Posted on:
Location Type: Hybrid
Location: New York City • New York • 🇺🇸 United States
Visit company websiteJob Level
JuniorMid-Level
Tech Stack
Python
About the role
- Design, build, and refine prompts that drive AI interviewer behavior, question sequencing, and responses
- Ensure prompts enforce role-specific requirements, structured interview flows, and consistent behavior
- Identify failure modes, edge cases, and sources of inconsistency in both conversations and evaluations
- Implement AI interviewer agents based on customer-specific role requirements, screening attributes, and interview questions
- Design and maintain AI-based evaluation and scoring systems aligned with defined rubrics and hiring criteria
- Ensure evaluations are accurate, consistent, and explainable at scale
- Build and own evaluation pipelines to measure conversation quality, scoring accuracy, and reliability
- Simulate real-world interview scenarios to validate interviewer behavior before and after changes
- Monitor production performance and iterate rapidly to improve outcomes across high-volume interviews
- Ensure AI interviewers perform reliably and efficiently in high-throughput environments
- Partner with engineering to balance quality, latency, and cost tradeoffs
- Contribute to guardrails around consistency, bias, and compliance as systems scale
- Work closely with engineering to integrate prompts and agent logic into production systems
- Partner with product and operations to translate customer requirements into precise AI behavior
- Help shape internal standards and best practices for prompt engineering, evaluation, and AI quality
Requirements
- 2+ years of experience working with LLMs, NLP systems, or AI agents in production
- Demonstrated experience designing and deploying prompts, agents, or AI-driven workflows that operate at scale
- Strong understanding of prompt engineering techniques, agent control, and failure modes
- Experience building or contributing to evaluation, testing, or QA frameworks for AI systems
- Comfort working cross-functionally with engineering, product, and operations to translate requirements into system behavior
- Strong analytical mindset with the ability to reason about accuracy, consistency, bias, and edge cases
- Bachelor’s degree in Computer Science, Engineering, Math, or a related technical field
- Familiarity with voice or conversational AI systems, including real-time or high-volume environments (preferred)
- Strong programming skills in Python (APIs, data pipelines, testing frameworks) (preferred)
- Hands-on experience with multiple LLMs (GPT, Claude, Gemini, LLaMA, Mistral, or fine-tuned models) (preferred)
- Experience designing multi-step AI agents with structured flows and state management (preferred)
- Experience operating AI systems in production and iterating based on real-world performance metrics (preferred)
Benefits
- Health insurance
- Flexible working hours
- Professional development opportunities
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
prompt engineeringNLP systemsAI agentsevaluation frameworksPythondata pipelinestesting frameworksmulti-step AI agentsreal-time AI systemshigh-volume environments
Soft skills
analytical mindsetcross-functional collaborationreasoning about accuracyreasoning about consistencyreasoning about biasreasoning about edge cases
Certifications
Bachelor’s degree in Computer ScienceBachelor’s degree in EngineeringBachelor’s degree in Math