Senior AI QA / AI Testing Specialist

Avvale

Quality Specialist ensuring AI agents function effectively in real-world scenarios at Avvale. Collaborating with product, data, and technology teams to create robust conversational experiences.

Posted 5/13/2026full-timeSão Paulo • 🇧🇷 BrazilSeniorWebsite

About the role

Key responsibilities & impact

Ensure the quality of conversational AI solutions.
Define and evolve the testing framework for AI agents and multichannel flows.
Validate the quality of conversational agents, including RAG and LLMs.
Structure quality metrics, dashboards, and acceptance criteria.
Review test coverage and ensure bug traceability.
Advise partners and squads on AI QA best practices.
Plan and execute statistical tests and performance experiments.

Requirements

What you’ll need

Senior experience in QA, focused on digital products or AI.
Experience testing conversational agents and multichannel flows (voice and text).
Proficiency in test strategy, acceptance criteria, and bug traceability.
Experience with statistical testing for AI, response evaluation, and conversational quality.
Knowledge of RAG, LLMs, and QA tools applied to AI.
Ability to structure quality metrics and dashboards.

Benefits

Comp & perks

Hybrid working arrangement; engagement on a contractor (PJ) basis.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

QAtest strategyacceptance criteriabug traceabilitystatistical testingresponse evaluationconversational qualityquality metricsdashboards

Soft Skills

advisingplanningexecution