Principal QA Engineer – AI Systems & Platform

HIVE

Principal QA Engineer responsible for building QA functions at Peach Pilot. Working on AI system in a remote capacity in Latin America.

Posted 4/13/2026contractRemote • 🇦🇷 ArgentinaLeadWebsite

Tech Stack

Tools & technologies

Python

About the role

Key responsibilities & impact

You will build and own the QA function at Peach Pilot — writing test code, designing eval pipelines, and setting the quality bar as we move from early-stage development into full production and enterprise deployment.
This is a fully remote contract role based in Latin America.
You will work directly with the US-based founding engineering team and must be available during US Eastern business hours with a minimum of 5 hours of daily overlap.
Establish the testing framework from zero: unit, integration, end-to-end, and LLM-specific evaluation pipelines
Define quality standards, test coverage requirements, and documentation practices in partnership with the VP of Engineering
Audit the existing platform and identify the highest-risk surfaces before the next major customer deployment
Validate the Enterprise Knowledge Graph for data accuracy, retrieval quality, and failure modes under real enterprise data conditions

Requirements

What you’ll need

7+ years of QA engineering experience with at least 3 years in a lead or senior role where you both wrote test code and owned quality outcomes
Hands-on experience testing LLM-powered applications
you understand prompt sensitivity, output variance, and how to build eval pipelines that catch regressions across model updates
You write test code. Python is your primary tool
Experience building and maintaining CI/CD-integrated test suites
Comfortable testing complex API chains, async/streaming responses, and multi-service workflows
Built or significantly improved a QA function in an early-stage or fast-moving environment
Strong English communication skills written and verbal
Available during US Eastern business hours with minimum 5 hours of daily overlap

Benefits

Comp & perks

Competitive contractor rate commensurate with experience
Paid monthly via Deel in USD

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

QA engineeringtest codeLLM testingeval pipelinesCI/CDAPI testingPythonunit testingintegration testingend-to-end testing

Soft Skills

leadershipcommunicationcollaborationproblem-solvingattention to detailorganizational skillsadaptabilitycritical thinkingtime managementstakeholder engagement