Develop and maintain test plans for multi-agent, LLM-driven features and workflows
Design and execute functional, integration, end-to-end, and regression tests for cloud-based AI services
Validate retrieval pipelines, embeddings, knowledge-base updates, and AWS Bedrock/serverless integrations
Build automated test suites (Python preferred) for backend workflows and microservices
Collaborate with developers to define quality criteria, ensure testability, and troubleshoot APIs, agents, and cloud infrastructure
Contribute to documentation for test cases, quality processes, and release readiness

Requirements

3+ years of QA or software engineering experience
Strong proficiency in Python scripting and experience working with BDD frameworks and tools including Pytest-BDD, Playwright, and Gherkin (or similar)
Experience with LLMs, model inference workflows, or prompt evaluation
Familiarity with RAG concepts, retrieval pipelines, embeddings, or vector search
Strong analytical thinker with excellent attention to detail
Clear communicator, comfortable working independently
Cloud Experience (Asset): Exposure to AWS Lambda or serverless applications, experience testing services integrated with AWS Bedrock, familiarity with cloud databases such as DynamoDB or Aurora, knowledge of S3, CloudWatch, IAM, and distributed service debugging; experience with Terraform or CloudFormation is a plus

Benefits

Employee & safety centric culture with the possibility of remote work
Flexible hours and scheduling available
Competitive total compensation along with generous vacation allowances
Vacation leaves
Wellness - onsite fitness center & company reimbursement for personal gym memberships
Internal Recognition
Employee Referral bonuses
Transit accessible with free onsite parking
Regular company events such as "Food Truck Fridays", Lunch & Learn and other planned activities

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

PythonBDD frameworksPytest-BDDPlaywrightGherkinLLMsmodel inference workflowsretrieval pipelinesembeddingsvector search

analytical thinkingattention to detailclear communicationindependence