Software Engineer II – Serverless Inference

DigitalOcean

Software Engineer II at DigitalOcean responsible for Serverless Inference infrastructure design and optimization. Collaborating on large-scale AI workloads focusing on throughput and fault tolerance.

Posted 7/6/2026full-timeBengaluru • 🇮🇳 IndiaJuniorMid-LevelWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and operating scalable, multi-tenant platforms and distributed systems, with a strong focus on reliability, observability, and operational excellence. Proficient in leveraging SRE principles and cloud-native architectures to enhance system performance and resilience.

Highest-signal resume keywords

Go / Golang ProgrammingKubernetes ManagementSRE PrinciplesDistributed Systems DesignObservability Proficiency

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Multi-Tenant Platform DevelopmentDistributed Backend SystemsOperational AutomationIncident ManagementCapacity PlanningPerformance DebuggingScalability OptimizationService OrchestrationReliability EngineeringCloud-Native Architecture

Soft Skills

CollaborationContinuous ImprovementOperational Discipline

Industry Keywords

AI InferenceIntelligent RoutingHigh-Scale Distributed ServicesProduction EnvironmentsObservability MetricsTime To First Token (TTFT)Time Per Output Token (TPOT)GPU Utilization

Tech Stack

Tools & technologies

CloudDistributed SystemsGoKubernetesMicroservices

About the role

Key responsibilities & impact

Design and build scalable, multi-tenant services that power AI inference and intelligent routing workloads.
Develop and operate high-scale distributed systems with strong reliability, availability, and performance goals.
Strengthen platform resiliency through improved observability, capacity management, automation, and operational tooling.
Partner closely with platform, GPU infrastructure, and product engineering teams to deliver production-grade systems and highly available APIs.
Raise the engineering bar through strong software design, operational discipline, incident management, and continuous improvement practices.
Contribute to architecture decisions around traffic management, service orchestration, reliability, and platform scalability.
Participate in on-call rotations and lead efforts to reduce operator pain, improve service health, and prevent recurring incidents.

Requirements

What you’ll need

2+ years of experience building and operating multi-tenant platforms or distributed backend systems
Strong experience operating high-scale distributed services in production environments
Deep understanding of SRE principles, including observability, incident management, reliability engineering, capacity planning, and operational automation
1+ years of hands-on experience with Go / Golang in production systems
1+ years of experience with Kubernetes
Strong understanding of cloud-native architectures, microservices, and distributed systems fundamentals
Experience debugging performance, scalability, and reliability issues in production systems
Observability Proficiency: Experience tracking infrastructure and inference metrics like Time To First Token (TTFT), Time Per Output Token (TPOT), and GPU utilization.

Benefits

Comp & perks

We innovate with purpose. You’ll be a part of a cutting-edge technology company with an upward trajectory.
We prioritize career development. You will work with some of the smartest and most interesting people in the industry.
We care about your well-being. Regardless of your location, we will provide you with a competitive array of benefits.
We reward our employees. The salary range for this position is based on market data, relevant years of experience, and skills.
DigitalOcean is an equal-opportunity employer.