Software Engineer

Baseten

Develop infrastructure and orchestration systems for deploying and managing large-scale distributed LLM inference; Work across the stack, from customer-facing features to low-level infrastructure components

Posted 6/2/2026Verified active Jul 25, 2026, 12:37 AMfull-timeSan Francisco • California • 🇺🇸 United StatesWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Candidates should emphasize their expertise in distributed systems and backend infrastructure, showcasing their ability to build and manage production systems with a focus on reliability and scalability. Strong collaboration and communication skills are essential for working effectively with cross-functional teams and improving developer experience.

Highest-signal resume keywords

Distributed SystemsBackend InfrastructureProduction Systems ManagementDeveloper ExperienceCollaboration and Communication Skills

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Large-Scale LLM InferenceKubernetesRoutingAutoscalingObservability

Soft Skills

Problem SolvingMotivation to LearnCollaborationCommunicationUser-Centric Thinking

Tools & Technologies

Infrastructure OrchestrationDeployment SystemsRuntime ManagementBenchmarking ToolsRelease Automation

Certifications & Qualifications

Bachelor's Degree in Computer ScienceMaster's Degree in EngineeringPh.D. in a Related Field

Industry Keywords

Model PerformanceInference EngineeringOperational ExcellenceTesting Best PracticesScalability Standards

Tech Stack

Tools & technologies

CloudDistributed systemsDistributed SystemsKubernetesSpark

About the role

Key responsibilities & impact

Baseten’s Inference Stack team builds the distributed runtime that powers large-scale LLM inference across our platform. We operate at the intersection of distributed systems, model performance, infrastructure, and developer experience. We enable customers to deploy and operate cutting-edge LLM models with industry-leading performance, scalability, reliability, and ease of use.
As a Software Engineer on the Inference Stack team, you’ll work across the stack
from the developer experience customers use to deploy models, the libraries used for features like tool calling and reasoning, all the way down to the systems we use to orchestrate deployments in Kubernetes and route traffic efficiently.
This is an ideal role for engineers who enjoy owning systems in production, solving hard integration problems, and making complex infrastructure simple and reliable for users.
Develop infrastructure and orchestration systems for deploying and managing large-scale distributed LLM inference
Work across the stack, from customer-facing features to low-level infrastructure components
Build platform capabilities related to routing, autoscaling, scheduling, observability, and runtime management
Improve the reliability, scalability, and usability of our inference stack
Collaborate closely with Model Performance engineers to make new inference optimizations broadly available to customers and easy to configure
Help define best practices around testing, release automation, benchmarking, and operational excellence

Requirements

What you’ll need

Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field
Strong background in distributed systems, backend infrastructure, or platform engineering
Experience building and operating production systems where reliability, latency, and scale are first-class concerns
Strong sense of developer experience: you think about how systems are used, not just how they work
Motivated and willing to learn new languages, frameworks, and systems as needed
Ability to debug complex systems across multiple layers of the stack
Genuine interest in inference engineering. You don’t need to have hands on experience but are willing to learn
Excellent communication and collaboration skills

Benefits

Comp & perks

Competitive compensation, including meaningful equity.
100% coverage of medical, dental, and vision insurance for employee and dependents
Flexible PTO policy including company wide Winter Break (our offices are closed from Christmas Eve to New Year's Day!)
Paid parental leave
Fertility and family-building stipend through Carrot
Company-facilitated 401(k)
Exposure to a variety of ML startups, offering unparalleled learning and networking opportunities.
Apply now to embark on a rewarding journey in shaping the future of AI! If you are a motivated individual with a passion for machine learning and a desire to be part of a collaborative and forward-thinking team, we would love to hear from you.
At Baseten, we are committed to fostering a diverse and inclusive workplace. We provide equal employment opportunities to all employees and applicants without regard to race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, genetic information, disability, or veteran status.
We are an Equal Opportunity Employer and will consider qualified applicants with criminal histories in a manner consistent with applicable law (by example, the requirements of the San Francisco Fair Chance Ordinance, where applicable).