QA Analyst, Mid-level
Mutant
full-time
Posted on:
Location Type: Hybrid
Location: São Paulo • Brazil
Visit company websiteExplore more
Tech Stack
About the role
- Design test strategies for conversational agents based on LLMs
- Develop test scenarios to validate agent behavior
- Automate tests for prompts and model responses
- Identify hallucinations, inconsistencies, and inappropriate responses
- Create test datasets and regression suites
- Validate adherence to guardrails and response policies
- Participate in quality analysis prior to deployment of new agent versions
- Work with technical teams to correct and improve agent behavior
- Monitor the quality of responses in production
Requirements
- Experience in Quality Assurance or Test Automation
- Experience creating test scenarios and test plans
- Ability to structure systematic tests for complex systems
- Experience with APIs and automated testing
- Hands-on experience with AI tools such as ChatGPT, Claude, Gemini
- Analytical and investigative mindset
- Strong attention to detail
- Experience with Generative AI or LLMs
- Experience testing chatbots or voicebots
- Knowledge of NLP or text analysis
- Experience with test automation in Python
- Experience in Customer Experience or Contact Centers
- Experience testing AI-based systems
Benefits
- Medical and dental care
- Life insurance
- Transportation allowance (commuter voucher)
- Meal and food vouchers
- Wellhub (Gympass)
- Childcare assistance
- Support for children with disabilities
- Performance-based bonus
- Birthday day off
- Alelo Multibenefits card
- Allya — partnerships and discounts via app
- Pet health insurance
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
Quality AssuranceTest AutomationTest ScenariosTest PlansAPIsAutomated TestingGenerative AILLMsNLPPython
Soft Skills
Analytical MindsetInvestigative MindsetAttention to Detail