LLM Engineer

• Collaborate closely with our AI and ML research teams to understand their infrastructure needs and obstacles
• Monitor and optimize the performance of our infrastructure ensuring high availability, scalability, and efficient resource utilization
• Help define and improve important measures of AI researcher efficiency, ensuring that our actions are in line with measurable results
• Collaborate with diverse teams, including researchers, data engineers, and DevOps professionals, to build a seamless and coordinated AI/ML infrastructure ecosystem
• Stay on top of the latest advancements in AI/ML technologies, frameworks, and effective strategies, and promote their implementation within the company

AI and ML Infra Software Engineer, GPU Clusters

• Design and implement retrieval and tool-calling flows using frameworks
• Ship APIs and workers in Python or TypeScript with streaming and backoff
• Develop chunking, embeddings, and indexing capabilities for various data types
• Create offline evaluations and metrics for success
• Implement safety, privacy, and compliance checks

Software Engineer, Artificial Intelligence/LLM – Multiple Seniority Levels

• Partner with a cross-functional team of engineers, research scientists, technical program managers, and product managers to deliver AI-powered products that change how our associates work and how our customers interact with Capital One.
• Design, develop, test, deploy, and support AI software components including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability, etc.
• Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more.
• Invent and introduce state-of-the-art LLM optimization techniques to improve the performance — scalability, cost, latency, throughput — of large scale production AI systems.
• Contribute to the technical vision and the long term roadmap of foundational AI systems at Capital One.

Senior Lead AI Engineer, FM Hosting, LLM Inference

• Identify and prototype emerging LLM inference use cases suited to heterogeneous hardware deployments.
• Build compelling proof-of-concept systems that demonstrate D-Matrix capabilities to customers, partners, and internal stakeholders.
• Develop and tune custom kernels and operator-level optimizations to maximize throughput and minimize latency.
• Drive quantization, sparsity, and batching strategies tailored to D-Matrix computational model.
• Build and maintain inference runtimes, serving frameworks, and evaluation tooling.
• Contribute to distributed inference systems: tensor/pipeline parallelism, disaggregated prefill/decode, KV-cache management.
• Work closely with hardware architects to provide firmware and compiler teams with actionable inference workload insights.
• Partner with product and business development to translate POCs into customer-facing demonstrations.
• Contribute to technical publications, whitepapers, and open-source projects that advance D-Matrix visibility.

Principal LLM Inference Engineer

• Define the resilience platform, own the product roadmap and delivery for specific platform features
• Collaborate with the open-source developer community — prioritizing GitHub issues, gathering feedback, supporting contributors, and channeling community signal back into the roadmap
• Collaborate with engineering on feature design, prioritization, execution, and architecture tradeoffs
• Align cross-functionally with other Product, Engineering, Product Marketing, and Field teams on requirements, roadmaps, messaging, and engagements

Technical Product Manager, AI Infrastructure Resilience

• lead the product marketing strategy for Cisco’s AI infrastructure portfolio
• connect technical capabilities to the business outcomes customers require for AI workloads and AI-native applications
• define the narrative, positioning, messaging, launch strategy, and sales enablement required to power secure, connected, AI-ready organizations
• partner with Product Management, Engineering, Sales, Corporate Marketing, Analyst Relations, and Communications
• translate sophisticated infrastructure innovation into market value
• drive cross-functional alignment
• equip the business to win with enterprise buyers and executives

Director of Product Marketing Strategy – AI Infrastructure

• Design and build the distributed, fault-tolerant infrastructure that powers Anrok's AI agent capabilities.
• Own technical architecture decisions for new agent infrastructure services end-to-end, from initial design through production deployment and iteration.
• Build the evaluation, observability, and reliability tooling that lets the rest of the team ship agents confidently on top of your infrastructure.
• Collaborate closely with product engineers and our tax data team to translate real compliance workflows into reliable, scalable agent primitives.
• Drive engineering best practices across the team: code quality, testing discipline, system design, and operational health.
• Identify and fix performance, reliability, and scalability bottlenecks as our agent systems grow.
• Help grow the team—mentoring engineers, contributing to technical culture, and raising the bar on how we build.

Software Engineer, Agentic AI Infrastructure

• Lead architecture for DPU-accelerated host networking, security, and network services.
• Define what runs on the DPU versus the host CPU, including tradeoffs around performance, cost, failure isolation, observability, debuggability, and operational complexity.
• Architect distributed firewalling, micro segmentation, virtual routing, service chaining, traffic steering, and line-rate policy enforcement.
• Guide platform decisions across DPU SDKs, programmable pipelines, Linux networking, control-plane services, telemetry, policy distribution, and upgrade models.
• Design APIs, policy models, configuration flows, state synchronization mechanisms, and recovery behavior for distributed networking systems.
• Build static-stability and failure-mode thinking into the architecture so the system behaves predictably during partial failures, control-plane loss, upgrades, rollbacks, and fleet-wide rollout.
• Provide technical leadership for AI cluster networking challenges, including congestion collapse, incast, ECMP imbalance, PFC/ECN behavior, noisy tenants, NIC firmware issues, switch misconfiguration, and GPU underutilization caused by network stalls.
• Align cross-functional stakeholders across hardware, firmware, product, and customer-facing teams to successfully translate complex architectural visions into shippable software.
• Elevate engineering standards across the organization through meticulous design reviews, proactively identifying systemic risks to scale, reliability, and security.
• Mentor senior engineering talent, accelerating team growth and technical depth in packet processing, host networking, and control-plane design.

Technical Leader, DPU Networking, AI Infrastructure

• Lead bring-up, validation, and debugging of large-scale AI clusters, infrastructure, and end-to-end workloads, setting the standard for how the team operates.
• Bring up, tune, and benchmark AI pre-training, post-training, and inference workloads using PyTorch, NeMo / Megatron, TensorRT-LLM, and adjacent NVIDIA AI software stacks.
• Profile and optimize end-to-end workload performance across compute, memory, networking, and communication layers using tools such as Nsight Systems, NCCL tests, and custom microbenchmarks.
• Analyze scaling efficiency for distributed LLM workloads using data, tensor, pipeline, and expert parallelism across modern GPU clusters, and translate findings into concrete tuning guidance.
• Own root-cause analysis of complex failures — hangs, performance regressions, topology sensitivity in large distributed environments.
• Define and build the resilience and failure-attribution stack: detecting, triaging, and attributing node, fabric, and workload failures across the cluster at scale.
• Build repeatable benchmark suites, automation, acceptance criteria, and qualification workflows on new platforms.
• Tune runtime settings, communication parameters, and deployment configurations in close partnership with framework, systems, and platform teams.
• Deliver actionable, data-driven recommendations based on profiling, benchmark results, and cluster characterization.
• Mentor engineers, drive technical standards, and act as a force multiplier across the broader performance and infrastructure organization.

Senior Software Engineer, DGX Cloud AI Infrastructure

• Bring up, validate, and debug large-scale AI clusters, infrastructure, and end-to-end workloads.
• Bring up, tune, and benchmark AI pre-training, post-training, and inference workloads using PyTorch, NeMo / Megatron, TensorRT-LLM, and adjacent NVIDIA AI software stacks.
• Perform root-cause analysis of failures in large distributed environments.
• Contribute to the resilience and failure-attribution tooling that detects, triages, and attributes node, fabric, and workload failures across the cluster.
• Build and maintain repeatable benchmark suites, automation, acceptance criteria, and qualification workflows on new platforms.
• Tune runtime settings, communication parameters, and deployment configurations in close partnership with framework, systems, and platform teams.
• Deliver actionable, data-driven recommendations based on profiling, benchmark results, and cluster characterization.

Software Engineer, DGX Cloud AI Infrastructure

• architect and scale foundational platforms that enable AI-driven silicon development across GPU and system-level design teams.
• impact how efficiently large-scale simulation, design workflows, and machine learning workloads are executed.
• lead the development of distributed, data-intensive infrastructure and data platforms that integrate directly with engineering workflows.
• drive cross-functional collaboration with hardware, software, and system teams to integrate AI capabilities.
• accelerate scalable model training, data processing, and deployment within the silicon development lifecycle.
• architect and deploy scalable AI/HPC infrastructure (on-prem, cloud, or hybrid) optimized for EDA workflows.
• develop and operationalize MLOps pipelines for model training, validation, and deployment integrated with semiconductor design and verification flows.
• build and manage large-scale data pipelines to ingest, process, and serve engineering datasets.

Senior Staff Engineer, AI Infrastructure

• Design, develop and maintain Generative AI applications, advanced AI models and conversational agents.
• Use Gen AI techniques like Retrieval Augmented Generation (RAG), Prompt Engineering, Embeddings etc. to enhance model outcomes.
• Perform evaluation on LLM output using appropriate metrics and iteratively improve results.
• Stay updated with the latest industry trends and advancements in AI to implement cutting-edge solutions.
• Communicate model results and solicit feedback from stakeholders at various levels.

LLM Engineer jobs in California

See all jobs on JobTailor