Computational Linguist, AI Evaluation

Twelve Labs

Vital member of ML Data Team designing evaluation frameworks and enhancing data quality. Collaborating with cross-functional teams to push the bounds of multimodal AI.

Posted 5/7/2026full-timeSan Francisco • California • 🇺🇸 United StatesMid-LevelSenior💰 $150,000 - $160,000 per yearWebsite

Tech Stack

Tools & technologies

Python

About the role

Key responsibilities & impact

You will be a vital member of our ML Data Team.
Manage resource allocation and timelines, adjusting direction flexibly based on real-time information across all data streams in your product vertical.
Enhance dataset and process quality through seamless collaboration with vendors and outsourcing partners.
Establish labeling guidelines, monitor data quality, and improve tools and infrastructure to build a sustainable data operations framework.
Partner with Engineering and AI Model teams to align on top priority data needs, design tools such as analytical reports and dashboards, and clearly communicate project progress.

Requirements

What you’ll need

5+ years of experience working in an AI focused data operations organization.
A proven track record designing and executing large scale data or evaluation projects, including gathering, labeling, and post-processing data.
The ability to analyze messy and complex data, identify overarching patterns, and distill your findings into crisp annotation guidelines or model quality reports.
Proficiency with Python, LLMs, or other popular industry tools for automation.
Excellent communication and project management skills, and the ability to support several projects simultaneously.
A foundational understanding of and interest in LLMs/VLMs and multimodal AI.
Conviction that data is the key ingredient for the performance and assessment of AI models.

Benefits

Comp & perks

Full health, dental, and vision benefits.
Flexible PTO and parental leave policy. Office closed the week of Christmas and New Years.
An open and inclusive culture and work environment.
Work closely with a collaborative, mission-driven team on cutting-edge AI technology.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

data operationsdata qualitydata labelingdata analysisPythonLLMsmultimodal AIproject managementautomationdata evaluation

Soft Skills

communicationcollaborationflexibilityorganizationproblem-solvingtime managementleadershipadaptabilitycritical thinkingattention to detail