Software Engineering, DevOps AI Rater/Evaluator
LILT AI
contract
Posted on:
Location Type: Remote
Location: Saudi Arabia
Visit company websiteExplore more
Tech Stack
About the role
- Evaluate AI outputs related to software engineering, DevOps, and infrastructure topics
- Perform structured scoring, comparison, classification, and judgment tasks
- Assess technical correctness, completeness, security implications, and best-practice alignment
- Identify hallucinations, incorrect code, unsafe recommendations, or misleading system guidance
- Apply domain-specific engineering and DevOps guidelines consistently across tasks
Requirements
- Software engineers, site reliability engineers, DevOps engineers, or platform engineers
- Experience with production systems, CI/CD pipelines, cloud infrastructure, or distributed systems
- Strong attention to detail and comfort working with structured evaluation criteria
- Deep domain expertise in software engineering, DevOps, or infrastructure
- Strong technical judgment and ability to apply criteria consistently
- Ability to explain reasoning clearly, especially in complex or high-risk technical scenarios
Benefits
- Native or professional fluency in one or more supported languages is required
- English fluency is required for guidelines, feedback, and collaboration
- Opportunities for recurring work based on performance and demand
- Compensation communicated upfront per project or task type
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
software engineeringDevOpsinfrastructureCI/CD pipelinescloud infrastructuredistributed systemstechnical judgmentstructured evaluationbest-practice alignmentsecurity implications
Soft Skills
attention to detailclear reasoningcommunicationconsistency in applicationevaluation criteria comfort