Software Engineer – BIS, Baseten Inference Stack

Baseten

Software Engineer role focused on building infrastructure for large-scale distributed LLM inference. Collaborating on deployment and orchestration systems at AI company Baseten.

Posted 6/2/2026full-timeSan Francisco • California • 🇺🇸 United StatesMid-LevelSenior💰 $180,000 - $360,000 per yearWebsite

Tech Stack

Tools & technologies

Distributed SystemsKubernetes

About the role

Key responsibilities & impact

Develop infrastructure and orchestration systems for deploying and managing large-scale distributed LLM inference
Work across the stack, from customer-facing features to low-level infrastructure components
Build platform capabilities related to routing, autoscaling, scheduling, observability, and runtime management
Improve the reliability, scalability, and usability of our inference stack
Collaborate closely with Model Performance engineers to make new inference optimizations broadly available to customers and easy to configure
Help define best practices around testing, release automation, benchmarking, and operational excellence
Debug complex production systems spanning Kubernetes, distributed runtimes, networking, and GPU workloads
Make thoughtful engineering tradeoffs balancing performance, reliability, operational simplicity, and developer experience
Own projects end-to-end: from architecture and implementation through deployment, monitoring, and iteration based on customer feedback

Requirements

What you’ll need

Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field
Strong background in distributed systems, backend infrastructure, or platform engineering
Experience building and operating production systems where reliability, latency, and scale are first-class concerns
Strong sense of developer experience: you think about how systems are used, not just how they work
Motivated and willing to learn new languages, frameworks, and systems as needed
Ability to debug complex systems across multiple layers of the stack
Genuine interest in inference engineering. You don’t need to have hands on experience but are willing to learn
Excellent communication and collaboration skills.

Benefits

Comp & perks

Competitive compensation, including meaningful equity.
100% coverage of medical, dental, and vision insurance for employee and dependents
Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
Paid parental leave
Fertility and family-building stipend through Carrot
Company-facilitated 401(k)
Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

distributed systemsbackend infrastructureplatform engineeringKubernetesinference engineeringrelease automationbenchmarkingdebuggingscalabilityobservability

Soft Skills

developer experiencecommunicationcollaborationproblem-solvingmotivationwillingness to learnoperational excellenceengineering tradeoffscustomer feedback iterationreliability focus

Certifications

Bachelor's degreeMaster's degreePh.D. in Computer SciencePh.D. in Engineering