Codvo.ai

Principal Engineer, Test

Codvo.ai

full-time

Posted on:

Origin:  • 🇬🇧 United Kingdom

Visit company website
✨ AI Apply
Manual Apply

Job Level

Lead

Tech Stack

AzureCloud.NET

About the role

  • Perform functional, performance, and security testing on cloud-native services deployed on Microsoft Azure.
  • Design and implement automated test suites for APIs, service components, and AI pipelines.
  • Automate the evaluation of AI system outputs to ensure accuracy, consistency, and safety of responses.
  • Collaborate with developers and data scientists to establish service-level quality metrics and observability hooks.
  • Validate services against AI regulatory frameworks and ensure traceability, fairness, and robustness in outcomes.
  • Participate in threat modelling and security validation of exposed APIs and AI services.
  • Provide feedback early in the lifecycle to reduce defects and improve design.
  • Mentor junior testers, encourage continuous learning, and contribute to a culture of innovation.

Requirements

  • 12+ years of experience in software testing, QA, or validation roles for cloud-native applications using Microsoft and .NET technologies.
  • Proficient in designing automated testing frameworks.
  • Hands-on experience with Azure DevOps, CI/CD pipelines, and containerized test environments.
  • Strong understanding of API testing, performance profiling, and security testing (including OWASP top 10).
  • Familiarity with LLM evaluation techniques, output scoring, and validation frameworks.
  • Understanding of prompt engineering, RAG, model orchestration, and hallucination detection.
  • Experience in testing for accuracy, relevance, and consistency of AI model predictions/generations.
  • Experience defining performance metrics for AI services and testing for the same.
  • Awareness of AI safety, bias detection, and explainability techniques.
  • Experience ensuring compliance with AI regulations and standards (e.g., NIST AI RMF, EU AI Act).
  • Strong belief in ethical AI practices, transparency, and end-user trust.
  • Excellent problem-solving skills, with the ability to analyse complex technical challenges and propose scalable solutions.
  • Experience working in Agile teams and collaborating across global R&D locations.
  • Demonstrated ability to mentor junior team members and foster a culture of continuous learning and innovation.