QA Engineer

• Own the evaluation stack. Design and build LLM-as-judge systems, calibrate them against human labels, and make agent quality measurable per-agent and per-failure-mode.
• Make the release gate real. Build per-PR eval harnesses and regression detection wired into CI, so quality is enforced automatically, not by manual passes.
• Build user simulators to generate test coverage and adversarial cases before real users hit them.
• Turn production signal into improvement - pipe real failures back into evaluation sets so the system compounds over time.
• Partner with product to turn "what good looks like" into concrete, measurable criteria.
• Grow into agent development - contribute to building and hardening the agents themselves, starting with the components you know most deeply from evaluating them.

Senior AI Backend Engineer – Agent Evaluation, Quality

• Participate and review requirements, specifications during sprint planning and provide any extra information or details which will ensure test coverage.
• Build, recommend and maintain appropriate frameworks to automate test scenarios.
• Creating detailed, comprehensive and well-structured test plans and test cases.
• Convert manual regression scenarios to automated test cases.
• Estimating, prioritizing, planning and coordinating testing activities during SDLC.
• Ensure that automated tests are integrated and executed reliably in the CI/CD environments.
• Work as a member of an Agile SCRUM team delivering high-quality software in two-week sprints
• Identify, troubleshoot, and document issues thoroughly during bugs tracking.
• Perform functional and non-functional testing to support the development team.
• Ability to think out-side-box for reproducing, re-creating issues.
• Ability to act as stand-alone and define tasks to start QA efforts in an Agile team.
• Document software issues and enhancement requests and assist development with concise and detailed steps for duplication of the defects found while performing manual functional, integration, and regression tests.

QA Engineer jobs in Saudi Arabia

See all jobs on JobTailor

Senior AI Backend Engineer – Agent Evaluation, Quality

Senior QA Backend Engineer