Senior Technical Program Manager – AI Tooling & Systems

Deepgram

Senior Technical Program Manager overseeing large-scale ML infrastructure projects at Deepgram. Collaborating with cross-functional teams for optimizing AI technologies and model deployment.

Posted 7/9/2026full-timeRemote • 🇺🇸 United StatesSenior💰 $152,000 - $190,000 per yearWebsite

Tech Stack

Tools & technologies

AWSAzureCloudGoogle Cloud PlatformPyTorch

About the role

Key responsibilities & impact

Own end-to-end delivery of AI infrastructure programs—from model training pipelines and experiment tracking to inference serving and production monitoring
Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling (e.g., vector databases, model servers, evaluation frameworks, prompt engineering platforms)
Serve as connective tissue between ML research, ML engineering, product, and data teams to align on ML system requirements, capability roadmaps, and deployment timelines
Drive cost and latency optimization for real-time inference workloads at scale
Build lightweight internal tools and processes to accelerate ML iteration cycles (experiment tracking, model versioning, A/B testing infrastructure)
Identify and resolve technical bottlenecks in training pipelines, serving infrastructure, and model evaluation workflows
Work closely with ML practitioners to translate research breakthroughs into scalable, observable systems

Requirements

What you’ll need

5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling (or equivalent)
Strong technical acumen in ML systems—ideally hands-on experience as an ML engineer, systems engineer, or ML infrastructure engineer
Experience coordinating cross-functional ML programs (e.g., model training → evaluation → serving → monitoring)
Proven ability to translate ML/research requirements into robust, scalable infrastructure
Comfortable working in ambiguity and helping teams navigate complex technical tradeoffs (e.g., accuracy vs. latency vs. cost)
Excellent communication with both technical and non-technical stakeholders
Familiarity with high-growth or startup environments
It Would Be Great If You Had
Hands-on experience with model serving frameworks (vLLM, TensorRT, TorchServe, or similar)
Experience optimizing LLM or speech/audio model inference (quantization, distillation, KV-cache optimization, batching strategies)
Familiarity with ML experiment tracking and versioning tools (MLflow, Weights & Biases, DVC, or similar)
Background in feature stores, vector databases, or real-time ML systems
Knowledge of cost optimization for GPU/ML workloads on cloud and on-premise infrastructure
Experience with multi-region model serving or edge deployment
Hands-on with relevant frameworks (PyTorch, CUDA, Hugging Face, etc.) or cloud platforms (AWS SageMaker, GCP Vertex AI, Azure ML)

Benefits

Comp & perks

Offers Equity
Offers Bonus
10% Annual Bonus

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Model Training PipelinesExperiment TrackingInference ServingModel VersioningA/B Testing InfrastructureML Systems ArchitectureLatency OptimizationQuantization TechniquesDistillation MethodsBatching Strategies

Soft Skills

Excellent CommunicationNavigating Ambiguity