Own Tracer end-to-end (internal tooling + systems)
Own the roadmap and execution for Tracer as an internal platform
Define Tracer’s success metrics: trace coverage, evaluation throughput, quality signals, reliability improvements, and time-to-debug
Ensure Tracer supports fast iteration cycles while protecting customer experience and production stability
Build systems that make prompt changes measurable, testable, and easy to roll out safely
Support workflows for customer-specific prompt logic and controlled deployments
Partner with engineering and customer teams to ensure prompt/tooling updates can be made quickly without distracting senior leadership
Create consistent QA processes and evaluation workflows (human review, automated evals, regressions, guardrails)
Standardize debugging workflows for failures and production incidents
Establish quality benchmarks that align with customer expectations and core product performance
Partner with engineering to ship tooling improvements quickly
Write scripts, prototypes, or small internal tools to accelerate iteration
Work closely with engineers on tracing infrastructure, analytics, dashboards, and experimentation systems
Partner with Product, Engineering, Ops, CS, and Sales to ensure Tracer drives real business outcomes
Translate feedback from customer-facing teams into tooling enhancements
Own internal stakeholder alignment and adoption for Tracer workflows

Requirements

5+ years of Product Management experience (or equivalent) in B2B SaaS, AI products, developer tools, workflow automation, or platform teams
Strong technical fluency and ability to partner deeply with engineering
Ability to write code (Python/Typescript/SQL or similar) to prototype, debug, or build internal tools
Experience operating with high ownership in fast-moving, ambiguous environments
Strong analytical skills and comfort defining metrics and evaluation systems
Excellent communication and ability to drive cross-functional alignment
Experience with LLM workflows, agent systems, prompt versioning, evaluation, or reliability tooling (preferred)
Experience building observability or debugging tooling (e.g., tracing, logging, dashboards, experiment tracking) (preferred)
Experience in healthcare or enterprise systems where reliability and trust are critical (preferred)
Strong customer empathy and ability to translate real-world workflows into platform improvements (preferred)

Benefits

equity
benefits as part of our total compensation package

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard skills

PythonTypescriptSQLdebuggingprototypingworkflow automationevaluation systemsobservability toolingdebugging toolingLLM workflows

Soft skills

analytical skillscommunicationcross-functional alignmentcustomer empathyownershipadaptabilitycollaborationproblem-solvingleadershiporganizational skills