Lead Machine Learning Engineer

ASAPP

Lead Machine Learning Engineer building and evaluating agentic and LLM-based systems at ASAPP, an AI-powered customer-experience company. Designing production ML evaluation frameworks for quality, safety, and performance at scale.

Posted 8/7/2026full-timeNew York City • California, New York • 🇺🇸 United StatesSenior💰 $170,000 - $190,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in designing and developing ML evaluation systems, particularly for agentic and LLM-based architectures, while effectively collaborating with cross-functional teams to translate research into impactful AI solutions. Proficient in mentoring engineers and ensuring quality and performance in complex systems.

Highest-signal resume keywords

Deep Experience In NLPProduction Experience With PythonArchitectural Skills In Complex Software SystemsExperience With LLM-Centric ServicesFamiliarity With Large-Scale ML Experimentation

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Machine LearningNatural Language ProcessingLLM-Based SystemsSoftware ArchitectureModel Optimization TechniquesTechnical Solution ImplementationEvaluation Framework DesignBenchmarkingSimulation FrameworksAgentic Systems Evaluation

Soft Skills

MentorshipCollaborationKnowledge SharingFeedback ProvisionWillingness To Learn

Tools & Technologies

AWSKubernetesDockerCI/CDKafkaAthena

Certifications & Qualifications

Bachelor’s Degree In Computer Science

Industry Keywords

AI SolutionsProduction-Grade SystemsTechnical DiscussionsQuality AssurancePerformance Evaluation

Tech Stack

Tools & technologies

AWSDockerKafkaKubernetesPython

About the role

Key responsibilities & impact

Design and develop ML evaluation systems for agentic and LLM-based architectures
Translate research ideas into production-grade ML systems with measurable impact
Partner with Research, Product, and Platform teams to productize experiments into robust AI solutions
Stay current with advancements in ML, NLP, and LLM systems and contribute to technical discussions
Mentor and support other engineers through design reviews, feedback, and knowledge sharing
Build and evaluate the core intelligence behind agentic AI systems
Design and own evaluation frameworks ensuring quality, safety, and performance across complex agentic systems

Requirements

What you’ll need

Deep experience in NLP and modern ML systems, including hands-on experience with LLM-based or agentic systems
Strong architectural skills and proven experience designing complex software systems
Production experience with Python, AWS, Kubernetes, and/or Docker
Experience implementing technical solutions and tackling challenging engineering problems
Bachelor’s Degree in CS or another related field
Experience building and evaluating agentic systems at scale
Production experience with LLM-centric services, including inference, orchestration, evaluation, and monitoring
Familiarity with large-scale ML experimentation, benchmarking, or simulation frameworks
Knowledge of techniques for optimizing model architectures for faster inference
Experience with AWS, CI/CD, Kafka, and Athena
Proficiency in technical mentorship of junior and mid-level engineers
Willingness to learn, collaborate across teams, and teach others

Benefits

Comp & perks

Competitive compensation with stock options
Comprehensive medical, vision, and dental insurance
401k matching
Fitness and wellness stipend
Mental well-being benefits
Professional learning and development stipend
Parental leave, including adoptive and foster parents
3 weeks paid time off (increases with tenure)
Sick leave
Bereavement leave
Jury duty leave
Performance bonus
Equity grant comprised of stock options