Senior Quality Engineer II – AI, Data
Salla E-Commerce Platform
full-time
Posted on:
Location Type: Hybrid
Location: Makkah • Saudi Arabia
Visit company websiteExplore more
Job Level
About the role
- Data Pipeline Validation: Test the integrity of data ingestion and transformation processes to ensure models are trained on high-quality datasets.
- API & Model Testing: Conduct deep API testing using tools like APIDog to validate model responses, handling complex edge cases such as zero-value parameters or unauthorized access.
- Generative Model Output Accuracy: Define and implement evaluation frameworks to measure the quality of generative AI outputs — including relevance, factual accuracy, coherence, and hallucination rate — using techniques such as LLM-as-a-judge, human-in-the-loop scoring, and automated benchmark suites.
- AI Agent Response Validation: Design test strategies to assess the correctness and reliability of agentic AI responses, covering tool-call accuracy, multi-step reasoning integrity, goal completion rate, and graceful failure handling for out-of-scope or adversarial inputs.
- Security Testing: Proactively identify vulnerabilities like request interception and price manipulation to safeguard the system including prompt injection and jailbreak attempts specific to generative AI surfaces.
- UI Automation: Build and maintain scalable automation suites using low-code tools like mabl, aiming for high Mabl Test Coverage %.
- AI Tooling Advocacy: Champion the use of AI assistants like Jira Rovo for requirement analysis and test case generation to increase team velocity.
- Root Cause Analysis (RCA): Lead the RCA process for production bugs and developer-rejected tickets to continuously improve the "Definition of Done".
Requirements
- Senior Experience: 8+ years of relevant work experience
- Data Proficiency: Strong SQL skills and experience testing data-heavy applications or ML models.
- Automation Mastery: Proficiency in automation tools (e.g., Selenium, mabl, Playwright)
- Security Mindset: Understanding of OWASP principles and practical experience in security testing (request interception, label tracking).
- Domain Knowledge: Experience in e-commerce and critical customer-flow analysis is highly preferred.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
data pipeline validationAPI testingmodel testinggenerative AI evaluationsecurity testingUI automationroot cause analysisSQLautomation toolsOWASP principles
Soft Skills
leadershipadvocacyproblem-solvinganalytical thinking