AI Infrastructure Software Engineer

NVIDIA

AI Infrastructure Software Engineer at NVIDIA designing and improving infrastructure for large-scale AI training. Focused on pre-training, supervised fine-tuning, and reinforcement learning workloads.

Posted 7/7/2026full-timeBeijing • 🇨🇳 ChinaMid-LevelSeniorWebsite

Tech Stack

Tools & technologies

Distributed SystemsPython

About the role

Key responsibilities & impact

Create and implement the training infrastructure spanning pre-training, SFT, and RL post-training for Physical AI world foundation models
Develop and improve the pre-training and SFT pipelines — large-scale data loading, distributed training, and checkpointing — to achieve high throughput and scalability
Develop and improve the inference and evaluation stack, including the inference engine, inference/generation pipelines (which also support RL rollout), and evaluation pipelines
Use methods like continuous batching and KV-cache management to achieve high throughput and low latency
Build and improve the effective interaction and data flow among the RL system's roles (policy, rollout, reward, simulation) while investigating system-level optimization opportunities
Integrate and orchestrate simulation and robotics environments as RL environments — driving the simulation↔rollout↔training loop at scale
Build and refine the distributed training backend — sharding/parallelism, mixed precision, activation checkpointing, and memory/throughput optimization across many GPUs
Improve the efficiency, scalability, and resiliency of training and RL workloads — focusing on fault tolerance, fast/elastic restart, and throughput optimization under preemption and hardware failure
Define meaningful, actionable reliability and efficiency metrics to track and improve system reliability
Root cause, triage, and resolve failures from the application level down to the framework, GPU, and network/hardware level

Requirements

What you’ll need

5+ years developing software infrastructure for large-scale AI or distributed systems
Bachelor's degree or higher in Computer Science or a related technical field (or equivalent experience)
Strong debugging and triage skills across the stack — from AI application down to GPU/hardware behavior
Proven track record building and scaling large-scale distributed systems, ideally distributed training or inference
Hands-on experience with AI training and/or inference infrastructure — RL/post-training, training frameworks, or inference serving
Proficiency in Python (plus scripting), and solid software engineering practices: testing, defensive programming, version control, and CI
Excellent communication and collaboration skills; intellectual curiosity, problem-solving, and willingness

Benefits

Comp & perks

Competitive salaries
Comprehensive benefits package

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Distributed TrainingInference PipelinesCheckpointingContinuous BatchingKV-Cache ManagementMixed Precision TrainingActivation CheckpointingFault ToleranceThroughput OptimizationReliability Metrics

Soft Skills

Excellent CommunicationCollaboration SkillsIntellectual CuriosityProblem-SolvingWillingness to Learn

Certifications

Bachelor's Degree in Computer Science