You will design and implement scalable APIs for agent orchestration and learner interaction
You will build data pipelines that feed agent evaluation and continuous improvement cycles
You will create event streaming infrastructure for real-time learner interaction analysis
You will ensure data quality and accessibility for training and evaluating learning agents
You will improve data infrastructure for cost and performance in close coordination with Data and ML teams
Design and implement rigorous evaluation frameworks to measure agent performance and improve cycles
Develop LLM evaluation processes and perform error analysis to identify systemic improvements for agentic learning systems
Support instrumentation that makes evaluation relevant (metrics, traces/logs, and analysis workflows)
Participate in on-call rotation and incident response for learning platform reliability
Contribute to runbooks, postmortems, and reliability improvements as we evolve our operating model

Requirements

5+ years production Python experience at scale
AWS expertise (Lambda, DynamoDB, S3, Glue, and Athena)
Experience with a relational database such as MySQL, PostgreSQL, or Oracle
Experience with agentic AI architectures and evaluation frameworks
Data pipeline development (streaming and batch)
Infrastructure as Code experience (CDK, Terraform)
Production SaaS experience, including participation in on-call / incident response
Experience in GitHub Actions or similar CI/CD platforms
Experience communicating updates and resolutions to customers and other partners
Active user of AI development tools (GitHub Copilot, Cursor, Claude Code, Codex) with personal projects and evolution over the past 12 months
Demonstrated examples of AI-augmented productivity gains
Enthusiasm for pushing boundaries of AI-assisted engineering.

Benefits

High-quality, low-deductible medical insurance
Low to no-cost dental and vision plans
5 weeks of paid time off (plus almost a dozen paid holidays)
Employer-funded retirement
Free tuition program
Parental leave
Mental health and wellbeing resources

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Pythondata pipeline developmentevent streamingLLM evaluation processeserror analysisinfrastructure as CodeGitHub Actionsrelational databasesagentic AI architecturesproduction SaaS

Soft Skills

communicationincident responsecollaborationproblem-solvingreliability improvements