Senior Software Engineer, AI Systems – vLLM, MLPerf

NVIDIA

full-time

Posted on: 10/9/2025

Location Type: Hybrid

Location: Santa Clara • California • 🇺🇸 United States

✨ AI Apply

💰 $184,000 - $287,500 per year

Senior

Distributed SystemsPythonPyTorch

About the role

Design and implement highly efficient inference systems for large-scale deployments of generative AI models.
Define inference benchmarking methodologies and build tools that will be embraced across the industry.
Develop, profile, debug, and optimize low-level system components and algorithms to enhance the throughput and the latency for the MLPerf Inference benchmarks on the newest NVIDIA GPUs.
Productionize inference systems with uncompromised software quality.
Collaborate with researchers and engineers to productionize trending model architectures, inference techniques and quantization methods.
Contribute to the design of APIs, abstractions, and UX that make it easier to scale model deployment while maintaining usability and flexibility.
Participate in design discussions, code reviews, and technical planning to ensure the product aligns with the business goals.
Stay up to date with the latest advancements and come up with novel research ideas in inference system-level optimization, then translate research ideas into practical, robust systems.

Bachelor’s, Master’s, or PhD degree in Computer Science/Engineering, Software Engineering, a related field, or equivalent experience.
5+ years of experience in software development, preferably with Python and C++.
Deep understanding of deep learning algorithms, distributed systems, parallel computing, and high-performance computing principles.
Hands-on experience with ML frameworks (e.g., PyTorch) and inference engines (e.g., vLLM and SGLang).
Experience optimizing compute, memory, and communication performance for the deployments of large models.
Familiarity with GPU programming, CUDA, NCCL, and performance profiling tools.
Ability to work closely with both research and engineering teams, translating pioneering research ideas into concrete designs and robust code, as well as coming up with novel research ideas.
Excellent problem-solving skills, with the ability to debug sophisticated systems.
A passion for building high-impact software that pushes the boundaries of what’s possible with large-scale AI.

Benefits

equity
benefits 📊 Resume Score Upload your resume to see if it passes auto-rejection tools used by recruiters Check Resume Score

Tip: use these terms in your resume and cover letter to boost ATS matches.

PythonC++deep learning algorithmsdistributed systemsparallel computinghigh-performance computingML frameworksinference enginesGPU programmingCUDA

problem-solvingcollaborationcommunicationtechnical planningdebuggingcreativity

Bachelor's degreeMaster's degreePhD degree