QA Engineer

• Define GenAI-Specific QA Strategy
• Develop a QA framework tailored to GenAI systems and workflows.
• Design tests for:
• - Prompt behavior across varied inputs and user tasks
• - Hallucination detection
• - Factual consistency and groundedness
• Blend manual and automated test design for both deterministic and stochastic outputs.
• Collaborate with teams to obtain or create sample data with clear target outputs.
• Test Plan Ownership
• Own end-to-end test strategy and execution for GenAI-powered features.
• Ensure coverage across:
• - Diverse prompt phrasing, user intents, and failure modes
• - Multiple GenAI features (e.g., summarization, generation, classification)
• - High-risk, edge-case, and compliance-driven scenarios.
• Prompt Validation & Evaluation
• Lead the design and implementation of prompt and model evaluation protocols:
• - Alignment between user input and intended behavior
• - Output fluency, tone, and coherence
• - Clarity, coverage, and relevance of responses
• Use Golden datasets and benchmark prompts to establish evaluation baselines.
• Human-in-the-Loop (HITL) Evaluation
• Design and manage SME-driven review workflows.
• Facilitate structured reviews focused on:
• - Correctness/accuracy based on metrics and SME feedback
• - Capturing edge-case failures
• Reporting and KPIs
• Define and track QA effectiveness using metrics such as:
• - Pass rate for high-risk use-cases
• - HITL reviewer agreement rates and flagging critical issues
• - Use-case specific measures of “quality”
• Deliver clear, actionable dashboards and reports to leadership on AI quality, safety, and readiness

Senior Quality Engineer – AI, E-Learning

QA Engineer

QA Lead Automation

Manual Functional Tester

QA Associate / Engineer

QA Engineer