Customer Service & Support AI Rater, Evaluator
LILT AI
contract
Posted on:
Location Type: Remote
Location: Anywhere in North America
Visit company websiteExplore more
About the role
- Evaluate AI outputs related to customer service and support interactions
- Perform structured scoring, comparison, classification, and judgment tasks
- Assess accuracy, clarity, tone, helpfulness, and alignment with support best practices
- Identify hallucinations, misleading responses, policy violations, or unsafe guidance
- Apply domain-specific customer support guidelines consistently across tasks
- Validate and refine evaluation rubrics and edge-case handling
- Perform adjudication where raters disagree
- Conduct error analysis and qualitative reviews of model behavior
- Partner with LILT research, product, and customer teams on evaluation design
- Support red-teaming, policy alignment, and model readiness assessments
Requirements
- Customer support professionals, service operations specialists, or CX practitioners
- Experience handling customer inquiries, support workflows, or service escalation
- Strong attention to detail and comfort working with structured evaluation criteria
- Native or professional fluency in one or more supported languages is required
- English fluency is required for guidelines, feedback, and collaboration
Benefits
- Contract-based, flexible participation
- Project-based work with clear expectations and timelines
- Opportunities for recurring work based on performance and demand
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
structured scoringcomparisonclassificationjudgment taskserror analysisqualitative reviewsevaluation designmodel behavior assessmentpolicy alignmentred-teaming
Soft Skills
attention to detailcommunicationcollaborationcritical thinkingproblem-solving