FREE ACCESS
5,000–10,000 jobs/day
See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Tech Stack
Tools & technologiesPython
About the role
Key responsibilities & impact- Develop and maintain a Python-based automation framework for validating AI agents
- Design and execute automated tests for LLM responses, conversational flows, and end-to-end agentic scenarios
- Apply modern AI testing strategies and evaluation methods
- Integrate automated tests into CI/CD pipelines
- Collaborate closely with engineering, AI/ML, and product teams
Requirements
What you’ll need- Must have 3 years of UK address history due to immediate onboarding
- Strong Python development skills, particularly for automation (PyTest preferred)
- Solid background in test automation and framework design
- Experience testing or validating AI/LLM-based systems (agentic workflows, conversational AI, RAG pipelines, etc.)
- Understanding of AI evaluation concepts
- Experience implementing CI/CD pipelines and automated batch executions
- Strong understanding of source code management, Git workflows, and collaborative engineering practices
- Ability to design structured, scalable, and maintainable automation solutions
Benefits
Comp & perks- Flexibility to work remotely
- Commitment to employee skill development
- Supportive work culture
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
Pythonautomation frameworktest automationPyTestAI testing strategiesCI/CD pipelinesGit workflowsautomated testsscalable automation solutionsbatch executions
Soft Skills
collaborationcommunicationproblem-solvingorganizational skills