QA Engineer – Quality Engineer

Direct Solution

QA Engineer garantindo testes e avaliações de IA em plataforma de orquestração para implementações SAP. Construindo automação, golden datasets e quality gates para releases confiáveis.

Posted 8/22/2026full-timeSão Paulo • 🇧🇷 BrazilMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in test automation, particularly in building and maintaining automated test suites for AI-driven products. Proficient in evaluating LLM outputs and ensuring the reliability of CI/CD pipelines while effectively managing defect triage and testing processes.

Highest-signal resume keywords

Test AutomationPythonE2E Testing FrameworksCI/CD Pipeline ManagementLLM Evaluation

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Test AutomationPythonTypeScriptE2E TestingAPI TestingCI/CDDefect Severity AssessmentPerformance TestingLoad TestingSecurity Testing

Soft Skills

Self-DirectedProcess EstablishmentInsight Generation

Tools & Technologies

PlaywrightCypressTricentisWorksoftSAPSalesforceERPCRM

Industry Keywords

QASDETTest EngineeringHuman-in-the-LoopEvaluation DesignHallucination Detection

Tech Stack

Tools & technologies

CypressERPPythonTypeScript

About the role

Key responsibilities & impact

Build and maintain the platform’s automated test suite, including E2E, integration, and regression tests
Create mechanisms to evaluate responses generated by AI agents
Develop golden datasets, scoring criteria, and regression tests for LLM-generated artifacts
Determine whether changes to prompts or models have improved or degraded results
Define Quality Gates and the criteria that may block a release
Ensure the reliability of CI pipelines
Identify and fix flaky tests and slow pipelines
Perform exploratory and release testing
Triage defects, assign severity, and track fixes through final validation
Turn failure patterns identified during testing into insights for product and roadmap evolution

Requirements

What you’ll need

3+ years of experience in QA, SDET, or Test Engineering
Hands-on experience with test automation for products in production
Strong knowledge of Python and/or TypeScript
Experience with modern E2E testing frameworks such as Playwright, Cypress, or equivalent
Experience with API testing
Solid knowledge of CI/CD, including pipeline creation and maintenance
Ability to test systems with non-deterministic responses
Experience evaluating LLM or Machine Learning outputs, or the ability to develop a consistent approach to doing so
Ability to accurately assess defect severity and impact
Use of AI tools in day-to-day work
Self-directed, with the ability to establish processes in environments that do not yet have a defined model
Previous experience at early-stage companies, preferably having helped build the QA function or processes from the ground up
Experience evaluating LLM-based features or AI agents in production
Knowledge of evaluation design (evals), hallucination detection, and Human-in-the-Loop (HITL) review processes
Familiarity with enterprise software ecosystems, especially SAP, Salesforce, ERP, and CRM
Experience with SAP testing tools such as Tricentis or Worksoft
Experience with SAP testing cycles and UAT
Knowledge of performance, load, or security testing
Sufficient programming ability to not only identify a bug but also potentially fix it
Professional to fluent English

Benefits

Comp & perks

Opportunity to build and define the quality function from the ground up, establishing standards and processes
Work on a technically challenging problem: ensuring the quality of non-deterministic AI systems applied to large-scale enterprise projects
Direct impact on the quality of artifacts produced by agents before they are used in project deliveries worth millions of dollars
Direct access to Engineering, Product, and the founders
Competitive compensation
Significant equity participation