Senior Technical Program Manager – AI Tooling & Systems

Deepgram

Senior Technical Program Manager overseeing large-scale ML infrastructure projects at Deepgram. Collaborating with cross-functional teams for optimizing AI technologies and model deployment.

Posted 7/9/2026full-timeRemote • 🇺🇸 United StatesSenior💰 $152,000 - $190,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in managing end-to-end AI infrastructure programs, with a strong focus on ML systems, model serving, and cost optimization. Proven ability to collaborate across teams to translate research into scalable solutions while optimizing for performance and efficiency.

Highest-signal resume keywords

ML Infrastructure Program ManagementTechnical Leadership in ML SystemsModel Serving Frameworks ExperienceCost Optimization for GPU/ML WorkloadsCross-Functional Coordination in ML Programs

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

Model Training PipelinesExperiment TrackingInference ServingModel VersioningA/B Testing InfrastructureML Systems ArchitectureLatency OptimizationQuantization TechniquesDistillation MethodsBatching Strategies

Soft Skills

Excellent CommunicationNavigating Ambiguity

Tools & Technologies

MLflowWeights & BiasesDVCVLLMTensorRTTorchServePyTorchCUDAAWS SageMakerGCP Vertex AI

Industry Keywords

AI InfrastructureML PlatformsReal-Time ML SystemsFeature StoresVector Databases

Tech Stack

Tools & technologies

AWSAzureCloudGoogle Cloud PlatformPyTorch

About the role

Key responsibilities & impact

Own end-to-end delivery of AI infrastructure programs—from model training pipelines and experiment tracking to inference serving and production monitoring
Define technical architecture, integration patterns, and rollout strategies for new ML systems and tooling (e.g., vector databases, model servers, evaluation frameworks, prompt engineering platforms)
Serve as connective tissue between ML research, ML engineering, product, and data teams to align on ML system requirements, capability roadmaps, and deployment timelines
Drive cost and latency optimization for real-time inference workloads at scale
Build lightweight internal tools and processes to accelerate ML iteration cycles (experiment tracking, model versioning, A/B testing infrastructure)
Identify and resolve technical bottlenecks in training pipelines, serving infrastructure, and model evaluation workflows
Work closely with ML practitioners to translate research breakthroughs into scalable, observable systems

Requirements

What you’ll need

5+ years of program management or technical leadership in ML infrastructure, ML platforms, or AI tooling (or equivalent)
Strong technical acumen in ML systems—ideally hands-on experience as an ML engineer, systems engineer, or ML infrastructure engineer
Experience coordinating cross-functional ML programs (e.g., model training → evaluation → serving → monitoring)
Proven ability to translate ML/research requirements into robust, scalable infrastructure
Comfortable working in ambiguity and helping teams navigate complex technical tradeoffs (e.g., accuracy vs. latency vs. cost)
Excellent communication with both technical and non-technical stakeholders
Familiarity with high-growth or startup environments
It Would Be Great If You Had
Hands-on experience with model serving frameworks (vLLM, TensorRT, TorchServe, or similar)
Experience optimizing LLM or speech/audio model inference (quantization, distillation, KV-cache optimization, batching strategies)
Familiarity with ML experiment tracking and versioning tools (MLflow, Weights & Biases, DVC, or similar)
Background in feature stores, vector databases, or real-time ML systems
Knowledge of cost optimization for GPU/ML workloads on cloud and on-premise infrastructure
Experience with multi-region model serving or edge deployment
Hands-on with relevant frameworks (PyTorch, CUDA, Hugging Face, etc.) or cloud platforms (AWS SageMaker, GCP Vertex AI, Azure ML)

Benefits

Comp & perks

Offers Equity
Offers Bonus
10% Annual Bonus