Diabetic Specialist Nurse – AI Chatbot Evaluation
Mentis AI
contract
Posted on:
Location Type: Remote
Location: United Kingdom
Visit company websiteExplore more
Job Level
About the role
- Transcript Review & Evaluation: Review 20–25 chatbot transcripts (500–700 words each), assessing the quality and clinical appropriateness of the chatbot's responses to simulated diabetes-related patient interactions.
- Likert-Scale Scoring: Rate each transcript across various evaluation dimensions such as factuality, safety compliance, bias, completeness, and tone.
- Gold Standard Creation: Your evaluations will serve as the foundation for establishing gold-standard benchmarks for the chatbot's performance, directly shaping future model improvements.
Requirements
- Clinical experience: Minimum 1 year experience managing diabetic patients
- Domain knowledge: Familiarity with evidence-based diabetes care protocols, patient education, and clinical best practices for diabetes management.
- Language: Professional working English for written deliverables.
- Relevant skills: The ideal candidate is comfortable evaluating clinical content for accuracy, safety, and appropriateness, and can apply clinical judgement to assess AI-generated health guidance in a diabetes context.
Benefits
- Flexible Work Arrangements: Fully remote and asynchronous
- Competitive Compensation: Hourly compensation in line with level of clinical experience
- Professional Development: Gain hands-on experience in AI evaluation, data quality, and health informatics, with training provided on scoring methodologies.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
transcript reviewevaluationLikert-scale scoringclinical judgementcontent evaluation
Soft Skills
attention to detailanalytical skillscommunication