FREE ACCESS
5,000–10,000 jobs/day
See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
Tech Stack
Tools & technologiesAWSCloudGoogle Cloud PlatformPython
About the role
Key responsibilities & impact- Contribute to AI evaluation pipelines, including offline evals, production tracing, and feedback systems;
- Implement and maintain performance metrics (e.g., response quality, task success, reliability) using established frameworks;
- Help create and maintain evaluation datasets and test cases to identify regressions;
- Analyze results and propose incremental improvements to model and agent quality;
- Contribute to AI system components such as RAG pipelines, retrieval systems, and multi-step workflows within existing architectures;
- Write clean, maintainable Python code that integrates with LLM providers and internal services;
- Support improvements to system reliability, observability, and performance in production;
- Deliver well-scoped projects with guidance from more senior engineers;
- Break down tasks, make steady progress, and be proactive in unblocking yourself by asking for help when needed;
- Contribute to team excellence through code reviews, documentation, and knowledge sharing;
- Collaborate with cross-functional partners to ship user-facing features.
Requirements
What you’ll need- 2–5 years of professional software engineering experience;
- Experience contributing to production systems as part of a team;
- Exposure to AI/ML systems with a strong interest in LLM-powered products;
- Experience debugging systems, working with data, and iterating on performance;
- Proficiency in Python or a similar language;
- Strong understanding of LLM concepts (prompting, RAG, evaluation);
- Familiarity with backend systems, APIs, and cloud environments (e.g., AWS, GCP);
- Exposure to logging, monitoring, or debugging tools;
- Interest in learning tools like LangGraph, vector databases, and evaluation platforms.
Benefits
Comp & perks- Medical insurance;
- Dental insurance;
- Life, AD&D, and Disability Insurance;
- Natural Disaster Support Program;
- Wellness Apps;
- Paid Parental Leave;
- Paid Time off inclusive of holidays and sick time;
- Working Remotely Stipend;
- One time WFH Office Set-Up Stipend;
- Retirement Plan;
- Financial Planning;
- Learning & Development Budget.
ATS Keywords
✓ Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
PythonAI evaluation pipelinesperformance metricsevaluation datasetsdebuggingLLM conceptsRAG pipelinesretrieval systemsAPIscloud environments
Soft Skills
collaborationproactive problem solvingcode reviewsdocumentationknowledge sharing