Apply

Ready to go for it?

AI Apply speeds things up—apply directly if you prefer.

FREE ACCESS
5,000–10,000 jobs/day
JobTailor Logo

See all jobs on JobTailor

Search thousands of fresh jobs every day.

Discover
  • Fresh listings
  • Fast filters
  • No subscription required
Create a free account and start exploring right away.
Titan AI

Lead QA Engineer

Titan AI

Lead QA Engineer responsible for AI evaluation frameworks and automated test suites for fintech. Collaborating on quality control processes and compliance in a startup environment.

Posted 5/27/2026full-timeRemote • 🇺🇸 United StatesSeniorWebsite

Tech Stack

Tools & technologies
PythonSelenium

About the role

Key responsibilities & impact
  • Design and execute the evaluation framework for LLM and agentic AI outputs.
  • Write the assertions, define the behavioral contracts, and own regression baselines for model behavior.
  • Write and maintain the automated test suite for backend APIs and frontend surfaces.
  • Own performance and load testing for latency-sensitive inference paths.
  • Produce test artifacts, audit logs, and process documentation that meet SOC 2 Type II standards.
  • Work directly with Forward Deployed Engineering on client-side validation and production issue reproduction.

Requirements

What you’ll need
  • Seven or more years in software QA engineering, with at least two years personally testing AI or ML systems.
  • Fluency in Python and experience building automated suites using pytest, Playwright, or Selenium.
  • Hands-on experience with RAGAS, DeepEval, LangSmith, or comparable evaluation tooling.
  • Ability to trace a failure from the application layer to infrastructure.
  • Integrated QA gates into CI/CD pipelines and owned the process end to end.
  • Experience in fintech, banking, or another regulated environment is a strong advantage.
  • Familiarity with document processing pipelines, multi-agent architectures, RAG validation, or observability tooling such as Arize or Langfuse.

Benefits

Comp & perks
  • Competitive base and meaningful equity.
  • Remote (US). Occasional travel to client sites and team offsites.

ATS Keywords

✓ Tailor your resume
Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools
software QA engineeringPythonautomated testingpytestPlaywrightSeleniumperformance testingload testingRAGASDeepEval
Soft Skills
problem-solvingattention to detailcommunicationcollaborationprocess ownership
Certifications
SOC 2 Type II