Staff Software Engineer, Inference Runtime

Anthropic

Be the technical lead for Inference Runtime at Anthropic, guiding AI systems development and architecture. Collaborate on performance and scalability across various platforms in a growing AI team.

Posted 6/12/2026full-timeSan Francisco • California, New York • 🇺🇸 United StatesLead💰 $405,000 - $485,000 per yearWebsite

Tech Stack

Tools & technologies

AWSDistributed Systems

About the role

Key responsibilities & impact

Set technical direction for the team, owning the architecture and roadmap for the shared runtime of the inference serving stack
Own and evolve the accelerator-agnostic runtime
Drive efficient accelerator usage across GPU, TPU, and Trainium
Build the runtime's validation surface around partitioned builds
Act as a technical counterpart to Anthropic's central Infrastructure org
Mentor engineers on the team through design review and collaboration

Requirements

What you’ll need

Deep background in systems engineering or ML infrastructure
Real depth in at least one accelerator ecosystem (CUDA/GPU, TPU, or Trainium/AWS Neuron)
Significant software engineering experience in high-performance, large-scale distributed systems
A track record of defining and using engineering metrics
Experience driving technical alignment across organizational boundaries
Strong written and verbal communication skills

Benefits

Comp & perks

Competitive compensation
Optional equity donation matching
Generous vacation and parental leave
Flexible working hours
Lovely office space

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

systems engineeringML infrastructureCUDAGPUTPUTrainiumAWS Neuronhigh-performance systemslarge-scale distributed systemsengineering metrics

Soft Skills

technical directionmentoringdesign reviewcollaborationcommunication