Staff Quality Assurance Engineer

Pearson VUE

Staff Quality Assurance Engineer managing end-to-end quality for AI projects. Leading testing and evaluation of complex AI systems in hybrid work environment.

Posted 6/30/2026full-timeHoboken • New Jersey • 🇺🇸 United StatesLead💰 $130,000 - $150,000 per yearWebsite

Tech Stack

Tools & technologies

JavaJavaScriptTypeScript

About the role

Key responsibilities & impact

Define quality strategies and own test planning from requirements through release, partnering closely with cross-functional teams
Design and execute scalable functional, regression, and integration testing strategies
Apply hands-on automation framework expertise across UI, API, and backend layers to support major product initiatives
Test AI-driven functionality, including LLM agents, multi-agent workflows, and RAG pipelines
Validate AI outputs for accuracy, relevance, consistency, and edge cases across non-deterministic systems
Own evaluation frameworks (e.g., RAGAS), test datasets, and regression baselines for AI features
Stay current with the evolving landscape of LLM evaluation, agent testing, and AI observability, and translate emerging techniques into practical testing approaches
Lead QE team for your project by setting technical direction, defining quality strategy, and ensuring delivery confidence
Contribute to QE standards and frameworks within your project scope, establishing practices that other engineers can adopt
Integrate automated tests into CI/CD pipelines and partner with engineering leads to embed quality into the delivery process
Leverage data, risk signals, and production insights to continuously improve test coverage and effectiveness

Requirements

What you’ll need

8+ years of experience in Quality Engineering or Quality Assurance
Proven hands-on experience testing AI/LLM-based systems, including agents, multi-agent workflows, and RAG pipelines
Experience with AI evaluation approaches (e.g., RAGAS, LLM-as-judge, regression baselines) and non-deterministic testing strategies
Strong automation skills using Playwright or a comparable framework
Proficiency in JavaScript/TypeScript and/or Java
Solid experience with API testing and data validation
Demonstrated track record of leading quality end-to-end for complex projects, from strategy through execution
Ability to define and drive QE technical direction within a project or product area
Experience working in Agile delivery environments and contributing to fast-moving release cycles

Benefits

Comp & perks

This position is eligible to participate in an annual incentive program, and information on benefits offered is here.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Functional TestingRegression TestingIntegration TestingAPI TestingData ValidationTest PlanningTest Dataset EvaluationNon-Deterministic TestingAutomation with PlaywrightRAGAS Evaluation

Soft Skills

LeadershipCollaborationStrategic ThinkingCommunication