FREE ACCESS
5,000–10,000 jobs/day
See all jobs on JobTailor
Search thousands of fresh jobs every day.
Discover
- Fresh listings
- Fast filters
- No subscription required
Create a free account and start exploring right away.
AI Agent Evaluator
Hippocratic AIEvaluating Hippocratic AI’s clinical agents for symptom triage, diabetes medication titration, and travel health guidance. Providing safety-focused feedback to improve AI-driven healthcare tools.
Core Competencies
Role fitCore Competencies
Use this summary to align your resume positioning with the role.
Demonstrates expertise in clinical evaluation and feedback, particularly in managing diabetes and triage protocols. Proficient in assessing AI agent interactions and ensuring adherence to clinical guidelines while providing actionable insights for quality assurance.
Highest-signal resume keywords
Registered Nurse LicenseNurse Practitioner LicenseClinical Experience in Diabetes ManagementTriage Protocol KnowledgeClinical Quality Assurance Experience
ATS Keywords
Tailor your resumeApplicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills
Clinical EvaluationInsulin TitrationTriage ProtocolsClinical Reasoning AssessmentFeedback Delivery
Soft Skills
Strong Written CommunicationIndependent Work
Tools & Technologies
AI-Assisted Clinical ToolsDigital Health Platforms
Certifications & Qualifications
Board Certification (ANCC or AANP)
Industry Keywords
Travel MedicineOccupational HealthPre-Travel ConsultationCDC GuidelinesTelephone Triage
About the role
Key responsibilities & impact- Evaluate recorded and simulated AI agent interactions involving acute symptom triage and assess escalation and de-escalation decisions
- Review agent conversations related to insulin and diabetes medication titration for clinical accuracy and guideline adherence
- Evaluate agent-delivered travel health guidance, including pre-travel counseling, vaccination recommendations, and destination-specific risk assessment
- Identify clinical safety concerns, edge cases, and gaps in triage logic or clinical reasoning
- Score and annotate agent performance using structured evaluation rubrics
- Provide actionable written feedback to clinical and product teams to inform agent refinement
- Participate in case review and calibration sessions with other clinical SMEs
- Help develop and refine test scenarios for ongoing agent quality assurance
Requirements
What you’ll need- Registered Nurses: active, unrestricted RN License
- Nurse Practitioners: active, unrestricted Nurse Practitioner license in the United States, with current board certification (ANCC or AANP)
- Minimum of 5+ years of direct clinical experience in primary care, urgent care, telephone/virtual triage, or a comparable acute-care setting
- Clinical experience managing patients with diabetes, including insulin titration
- Strong, current working knowledge of relevant triage protocols and clinical practice guidelines
- Ability to critically evaluate clinical reasoning and identify inaccuracies or safety risks in written or recorded clinical interactions
- Comfortable working independently in a remote, asynchronous contract role
- Strong written communication skills for delivering structured clinical feedback
- Experience in telephone triage or nurse advice line settings (recommended)
- Experience in travel medicine, occupational health, or pre-travel consultation (recommended)
- Familiarity with CDC and other travel medicine guidelines (recommended)
- Prior experience in clinical quality assurance, utilization review, or case review (recommended)
- Familiarity with AI-assisted clinical tools or digital health platforms (recommended)
Benefits
Comp & perks- Fully remote work
- Flexible scheduling
- Hourly compensation commensurate with experience