Educational Technology AI Rater, Evaluator
LILT AI
contract
Posted on:
Location Type: Remote
Location: United States
Visit company websiteExplore more
About the role
- Evaluate AI outputs related to educational, instructional, and learning content
- Perform structured scoring, comparison, classification, and judgment tasks
- Assess pedagogical accuracy, clarity, appropriateness, and learning effectiveness
- Identify hallucinations, misleading explanations, factual errors, or unsafe educational guidance
- Apply domain-specific education and instructional guidelines consistently across tasks
- Validate and refine evaluation rubrics and edge-case handling
- Perform adjudication where raters disagree
- Conduct error analysis and qualitative reviews of model behavior
- Partner with LILT research, product, and customer teams on evaluation design
- Support red-teaming, educational quality review, and model readiness assessments
Requirements
- Educators, instructional designers, curriculum developers, or learning professionals
- Experience with teaching, curriculum design, assessment, or educational technology
- Strong attention to detail and comfort working with structured evaluation criteria
- Deep domain expertise in education, instructional design, or learning sciences
- Strong judgment and ability to apply criteria consistently
- Comfort working with structured evaluation workflows
- Ability to explain reasoning clearly, especially in instructional or learner-facing scenarios
- Reliability, professionalism, and respect for quality standards
- Native or professional fluency in one or more supported languages is required
- English fluency is required for guidelines, feedback, and collaboration
Benefits
- Contract-based, flexible participation
- Project-based work with clear expectations and timelines
- Opportunities for recurring work based on performance and demand
- Compensation communicated upfront per project or task type
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
curriculum designassessmenteducational technologyevaluation rubricserror analysisqualitative reviewspedagogical accuracystructured evaluation criteriainstructional designlearning sciences
Soft Skills
attention to detailstrong judgmentclear reasoningprofessionalismreliabilityrespect for quality standardsability to apply criteria consistentlycomfort with structured workflowscollaborationcommunication