Quality Assurance Engineer

Hupo

QA Automation Engineer for AI team at Hupo focusing on testing AI pipelines in banking and financial services. Collaborating to ensure release quality and compliance in a fast-paced environment.

Posted 7/27/2026full-timeRemote • 🇮🇳 IndiaMid-LevelSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in QA Automation Engineering with a focus on AI systems, including testing frameworks, CI/CD integration, and conversational AI evaluation. Proficient in validating voice and chat agents, ensuring high-quality outputs through rigorous testing methodologies.

Highest-signal resume keywords

QA Automation EngineeringConversational AI TestingCI/CD Pipeline DevelopmentLLM Evaluation ToolingDatabase Testing

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Automated TestingUI/E2E Test AutomationAPI Test AutomationLoad TestingPerformance TestingPrompt EngineeringRAG TestingSpeech-To-Text TestingText-To-Speech TestingRegression Strategy

Soft Skills

Builder MindsetPragmatic Process Approach

Tools & Technologies

PlaywrightLangfuseLangSmithDeepEvalRAGASGitCloud Knowledge

Industry Keywords

AI SystemsConversational AgentsAutomation FrameworksAgile Sprint LifecycleQuality Assurance Standards

Tech Stack

Tools & technologies

Cloud

About the role

Key responsibilities & impact

Build and maintain automated testing for AI voice and chat agents, from single conversational turns to full roleplay flows.
Test LLM output quality - correctness, consistency, structured output, language fidelity, prompt regression - using evaluation harnesses (LLM-as-judge, golden datasets, tolerance-based assertions for non-determinism).
Validate RAG pipelines: retrieval relevance, grounding/faithfulness, and answer quality.
Test voice pipelines: STT/TTS accuracy and real-time, low-latency behavior across languages.
Automate across UI/E2E, API, and AI output layers, and build CI/CD from scratch - lint, type-check, tests, evals, coverage gates, deployment checks.
Own release quality: regression strategy, catching breakages early, and clear go/no-go calls.
Cover database and load/performance testing; extend automation to desktop and mobile clients.
Take over and extend existing QA automation, and strengthen shared frameworks, tooling, and QA processes/standards.

Requirements

What you’ll need

5+ years as a QA Automation Engineer, with a proven track record testing AI systems - not just traditional software.
Builder mindset: established testing frameworks, standards, and CI/CD-integrated automation from scratch.
Hands-on experience testing conversational AI - voice and/or chat agents - on real, shipped projects.
Strong grasp of LLMs, prompt engineering, and RAG - designing tests for nondeterministic output and evaluating retrieval/generation quality.
Understanding of voice pipelines (speech-to-text, text-to-speech) and how to test them in automation.
Hands-on with LLM evaluation/observability tooling - e.g. Langfuse, LangSmith, DeepEval, RAGAS.
Proficient in UI/E2E (e.g. Playwright) and API test automation; experience building CI/CD pipelines from scratch.
Comfort with database testing, load/performance testing, and desktop/mobile test automation.
Solid Git discipline, basic cloud knowledge, and familiarity with the agile sprint lifecycle.
Track record in fast-paced, fast-shipping environments - ramps quickly on unfamiliar systems, pragmatic about process.

Benefits

Comp & perks

Flexible work arrangements