Johnson & Johnson

Senior Engineer – GenAI Quality Assurance

Johnson & Johnson

full-time

Posted on:

Location Type: Hybrid

Location: Barcelona • Spain

Visit company website

Explore more

✨ AI Apply
Apply

Job Level

Senior

Tech Stack

AWSAzureCloudGoogle Cloud PlatformPython

About the role

  • Design and implement automated and manual QA strategies for GenAI systems
  • Build regression tests, generate synthetic test data, build test harnesses and evaluation pipelines for accuracy, grounding, robustness, safety
  • Validate prompt logic, model behavior and evaluation metrics
  • Execute tests and debug pipelines
  • Support deployment of GenAI products
  • Lead system design reviews, advocating best practices for architecture and reliability of GenAI applications
  • Collaborate closely with data scientists and developers to embed testing and quality checkpoints
  • Establish testing standards and workflows for continuous integration and deployment
  • Conduct code reviews with a focus on design, testability, and maintainability
  • Document system design decisions, test cases, and best practices
  • Close collaboration with domain experts and stakeholders to validate developed solutions

Requirements

  • 3+ years of experience as a software engineer (preferably 5), or QA engineer in production systems, preferably including at least 1 year experience in GenAI systems
  • Strong software engineering background with QA ownership
  • Familiarity with RAG & Graph frameworks
  • Advanced Python skills
  • Experience working with LLM-based systems in real environments
  • Hands-on experience with: LLM APIs AWS Bedrock (Anthropic & OSS) Azure Foundry (OpenAI) GCP (Gemini) Cloud platforms (AWS, Azure, GCP)
  • Proven ability to implement test frameworks for GenAI applications
  • Knowledge of API testing, data validation, and data pipeline testing
  • Experience with version control (Git)
  • Experience with building and deploying AI/ML products through the full product lifecycle
  • Good understanding of scientific computing and ML frameworks
  • Solid understanding of safety, bias, and failure modes of GenAI systems
  • Comfortable working in a relatively fast-moving environment where priorities may shift and initiative is valued
  • Proactive problem-solver who can anticipate needs and find solutions
  • Comfortable working independently, but also a strong team player.
Benefits
  • flexible working arrangements
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
automated QA strategiesmanual QA strategiesregression testssynthetic test datatest harnessesevaluation pipelinesPythonAPI testingdata validationdata pipeline testing
Soft Skills
problem-solvingteam playerinitiativecollaborationcommunicationleadershipadaptabilityattention to detailproactivenessdesign review