Senior AI Engineer – Agentic AI Evaluation

Resaro

Senior Engineer for agentic AI evaluation at Resaro, responsible for developing the evaluation engine and ensuring the integrity of AI assessments.

Posted 7/14/2026full-timeMunich • 🇩🇪 GermanySeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates extensive experience in software engineering with a focus on agentic AI, including building and evaluating LLM agents. Possesses strong engineering judgment and the ability to translate complex governance into practical engineering solutions while ensuring clear communication and measurable outcomes.

Highest-signal resume keywords

Python ProgrammingAgentic AI DevelopmentLLM Agent EvaluationEngineering JudgmentClear Communication

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Software EngineeringProduction SystemsMulti-Turn Behaviour EvaluationTool-CallingMemory ManagementMulti-Step Control FlowModern InteroperabilityScoping and Decomposing ProblemsObservable Outcomes

Soft Skills

CommunicationCollaborationProblem-SolvingDocumentation

Industry Keywords

Agentic Evaluation EngineGovernance TranslationEmerging AutonomyLab-to-Production GapTrustworthy Evaluation

Tech Stack

Tools & technologies

Python

About the role

Key responsibilities & impact

Own the agentic-evaluation engine.
Build the capability that scores agents on the metrics that matter.
Evaluate stateful, multi-turn behaviour - not single prompts.
Make evaluation trustworthy.
Build the agentic-security surface.
Close the lab-to-production gap.
Translate governance into engineering.
Extend into emerging autonomy.

Requirements

What you’ll need

5+ years of professional software engineering, shipping and operating production systems (not prototypes) in Python.
Demonstrated depth in agentic AI - you've built, orchestrated, or rigorously evaluated LLM agents, and understand tool-calling, memory, multi-step control flow, and modern interoperability.
A real grasp of how agents are measured - and how measurement fails.
Strong engineering judgement - you can take an ambiguous, research-adjacent problem, scope it, decompose it, and drive it to a shipped, tested, observable outcome without close supervision.
A genuine bias to go deep and stay hands-on, paired with the communication to be the person others come to for this domain - you document decisions and leave work others can pick up.
Clear written and verbal communication, and comfort being measured against concrete quarterly outcomes, including across a distributed Singapore-Europe team.

Benefits

Comp & perks

Flexible work arrangements