Participate across the AI solution lifecycle from requirements and design through development, testing, and deployment
Build and maintain systems that exercise agentic workflows against realistic scenarios, edge cases, and failure conditions
Define contracts, validation, and explicit failure behavior at integration points
Create shared libraries, reference harnesses, and validation utilities that teams can reuse
Architect agentic systems so correctness, safety, and expected behavior are enforced by design
Participate in solution design to identify likely failure patterns in agentic systems
Define and implement cost-conscious evaluation strategies for AI-enabled systems
Work with cross-functional program teams to translate customer and mission needs into criteria
Communicate estimates, progress, and technical tradeoffs clearly to development leads and product owners

Requirements

5+ years of professional software engineering experience
Demonstrated experience building bespoke validation, evaluation, or testing frameworks for complex systems, using or expanding tools such as Playwright, pytest, k6, Guardrails AI, or similar frameworks
Data validation, schema enforcement, and contract testing experience in distributed or service-oriented architectures
Experience treating quality, reliability, and correctness as core design concerns
Experience defining, interpreting, and acting on quality signals to assess system readiness
Hands-on experience designing, building, and operating AI-powered systems, including agentic workflows or orchestrated LLM-based applications
Solid understanding of Amazon Web Services and its native AI/ML services
Practical experience integrating automation into CI/CD pipelines and delivering software in an iterative, agile environment
Working knowledge of security best practices relevant to regulated or mission-critical contexts
Proven ability to work effectively with cross-functional teams

Benefits

Paid Time Off
Paid Company Holidays
Medical, Dental & Vision Insurance
Optional HSA and FSA
Base and Voluntary Life Insurance
Short Term & Long-Term Disability Insurance
401k Matching
Employee Assistance Program

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

software engineeringvalidation frameworksevaluation frameworkstesting frameworksdata validationschema enforcementcontract testingAI-powered systemsCI/CD pipelinesagile development

Soft Skills

communicationcross-functional collaborationproblem-solvingdesign thinkingtechnical tradeoff analysis