Customer Service, Support AI Rater, Evaluator
LILT AI
contract
Posted on:
Location Type: Remote
Location: United States
Visit company websiteExplore more
About the role
- Evaluate AI outputs related to customer service and support interactions
- Perform structured scoring, comparison, classification, and judgment tasks
- Assess accuracy, clarity, tone, helpfulness, and alignment with support best practices
- Identify hallucinations, misleading responses, policy violations, or unsafe guidance
- Apply domain-specific customer support guidelines consistently across tasks
- Validate and refine evaluation rubrics and edge-case handling
- Perform adjudication where raters disagree
- Conduct error analysis and qualitative reviews of model behavior
- Partner with LILT research, product, and customer teams on evaluation design
- Support red-teaming, policy alignment, and model readiness assessments
Requirements
- Customer support professionals, service operations specialists, or CX practitioners
- Experience handling customer inquiries, support workflows, or service escalation
- Strong attention to detail and comfort working with structured evaluation criteria
Benefits
- Flexible participation
- Project-based work with clear expectations and timelines
- Opportunities for recurring work based on performance and demand
- Compensation communicated upfront per project or task type
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
AI evaluationstructured scoringerror analysisqualitative reviewsevaluation rubricsadjudicationcustomer support guidelinesmodel behavior assessmentred-teamingpolicy alignment
Soft Skills
attention to detailcommunicationcollaborationcritical thinkingproblem-solving