LLM Engineer

• Collaborate closely with our AI and ML research teams to understand their infrastructure needs and obstacles
• Monitor and optimize the performance of our infrastructure ensuring high availability, scalability, and efficient resource utilization
• Help define and improve important measures of AI researcher efficiency, ensuring that our actions are in line with measurable results
• Collaborate with diverse teams, including researchers, data engineers, and DevOps professionals, to build a seamless and coordinated AI/ML infrastructure ecosystem
• Stay on top of the latest advancements in AI/ML technologies, frameworks, and effective strategies, and promote their implementation within the company

AI and ML Infra Software Engineer, GPU Clusters

• Design, build, and deliver backend systems and APIs.
• Build and maintain full-stack product features.
• Provide technical leadership.
• Collaborate across disciplines.
• Support and own your software in production.
• Communicate clearly and multiply your impact.

Senior Software Engineer II – LLM Observability

• Design and build GenAI-powered developer tooling: prompts, IDE integrations, Cursor/Claude rules, and telemetry to understand how they’re used.
• Build and run the GenAI developer platform: Tools, services, and integrations leveraging LiteLLM to power both local and background/long-running agents, providing auth, configuration, and LLM routing.
• Own guardrails and observability for agentic workflows: integrate with Datadog, DX, and FinOut, define sensible standards, and make sure agents interact with Jira, GitHub, and internal platforms in a safe, auditable way.
• Work closely with engineers and managers across Product Engineering to understand how they build software today and to design solutions that actually fit the way they work.
• Support our mission and values, including journalistic independence, and keep ethics and safety central to how we use AI in developer workflows.
• This role includes limited on-call responsibilities; the schedule will be set when you join. Demonstrate support and understanding of our value of journalistic independence and a strong commitment to our mission to seek the truth and help people understand the world.
• You will report to the Engineering Manager of the DevEx Platforms team

Senior Engineer, Developer Platforms, Generative AI

• Build and integrate crucial public good infrastructure to support evidence-based AI-native data system implementation that strengthens data feedback loops.
• Lead a research and development portfolio to test, co-develop, and refine integrated data and AI infrastructure solutions in real-world education and workforce contexts.
• Design and lead iterative learning cycles at the intersection of the AI Infrastructure and System Feedback Loops (SFL) portfolios.
• Generate actionable evidence about what conditions, infrastructure configurations, and integrated approaches accelerate impact for learners.
• Apply mastery-level expertise in implementation research and education data systems, and a proactive, solutions-oriented perspective to navigate ambiguity and drive thorough, equity-centered inquiry in partnership with grantees, researchers, and cross-divisional colleagues.

Senior Program Officer, Data & AI Infrastructure Integration

• Design and implement retrieval and tool-calling flows using frameworks
• Ship APIs and workers in Python or TypeScript with streaming and backoff
• Develop chunking, embeddings, and indexing capabilities for various data types
• Create offline evaluations and metrics for success
• Implement safety, privacy, and compliance checks

Software Engineer, Artificial Intelligence/LLM – Multiple Seniority Levels

• Drive the on-site deployment, integration, and scaling of enterprise Generative AI solutions.
• Embed directly within customer engineering teams to operationalize Large Language Models (LLMs) and retrieval systems across multi-cloud environments (AWS, Azure, GCP).
• Bridge the gap between AI research and production-grade cloud infrastructure.
• Collaborate with cross-functional teams and business partners to leverage analytical skills and ensure business value.

Forward Deployed Engineer, Generative AI

• Define product strategy and roadmap for GPU instances, clusters, and services
• Manage the entire product lifecycle from planning to end-of-life
• Develop and execute go-to-market strategies, messaging, and launch plans
• Collaborate with ecosystem partners on roadmap and technical needs
• Translate AI, HPC and graphics workload needs into specifications and performance goals
• Oversee GPU infrastructure lifecycle components and implement data-driven decisions
• Identify enhancements for proactive monitoring and support processes

Senior Product Manager, AI Infrastructure, GPU

• Design, deploy, and optimize next-generation, customer-facing agentic workflows.
• Work on a unified platform bridging digital discovery and customer operations.
• Use Customer Experience Agent Studio (CX Agent Studio) to build multi-agent applications.
• Implement Code Agents leveraging programmatic SDKs.
• Write custom Python/TypeScript tool callbacks.
• Utilize localized CLI workflows to structure, lint, test, and sync complex agent hierarchies.

Conversational AI Engineer

• Partner with a cross-functional team of engineers, research scientists, technical program managers, and product managers to deliver AI-powered products that change how our associates work and how our customers interact with Capital One.
• Design, develop, test, deploy, and support AI software components including foundation model training, large language model inference, similarity search, guardrails, model evaluation, experimentation, governance, and observability, etc.
• Leverage a broad stack of Open Source and SaaS AI technologies such as AWS Ultraclusters, Huggingface, VectorDBs, Nemo Guardrails, PyTorch, and more.
• Invent and introduce state-of-the-art LLM optimization techniques to improve the performance — scalability, cost, latency, throughput — of large scale production AI systems.
• Contribute to the technical vision and the long term roadmap of foundational AI systems at Capital One.

Senior Lead AI Engineer, FM Hosting, LLM Inference

• Design and build production-grade generative AI systems - agentic workflows, multi-step RAG pipelines, and LLM-powered applications integrated with enterprise data and services
• Define and implement reusable engineering patterns for prompt management, workflow versioning, structured outputs, tool orchestration, and rollback across production AI services
• Apply judgment around model selection and routing, token and latency optimization, cost management, and the appropriate boundaries between AI-driven and deterministic application logic
• Continuously evaluate emerging AI models, tools, and architectural approaches, incorporating improvements into existing systems incrementally
• Integrate AI systems with enterprise data sources, internal APIs, and platforms to enable reliable, production-ready workflows
• Own operational outcomes for production AI systems - reliability, latency, throughput, cost efficiency, and scalability targets
• Implement and maintain monitoring, observability, tracing, and alerting frameworks to ensure operational visibility and rapid issue resolution
• Design and maintain CI/CD pipelines for deployment, versioning, and release management of AI services
• Lead production incident response and root cause analysis, driving systemic improvements that reduce recurrence
• Build and maintain automated evaluation pipelines for LLM outputs - prompt regression testing, retrieval quality validation, and failure mode tracking
• Implement human-in-the-loop controls, content guardrails, schema validation, and structured output enforcement to ensure trusted and auditable AI outputs
• Secure AI systems against prompt injection, data leakage, and unauthorized access, aligning with enterprise compliance and security standards
• Own the team's GenAI technical direction - defining and enforcing engineering standards, patterns, and best practices across all GenAI workstreams
• Make and defend architectural decisions with clarity, providing the technical rationale needed for the Manager and stakeholders to align and move forward confidently
• Work closely with the Manager, GenAI Engineering to receive, refine, and execute on scoped GenAI work - contributing technical judgment to prioritization and tradeoff decisions
• Provide hands-on code review and technical guidance to engineers contributing to GenAI workstreams, raising overall quality through direct feedback and demonstration
• Champion an iterative delivery culture - shipping incrementally, incorporating feedback, and improving continuously in a regular production release cadence

Generative AI Engineer

• Identify and prototype emerging LLM inference use cases suited to heterogeneous hardware deployments.
• Build compelling proof-of-concept systems that demonstrate D-Matrix capabilities to customers, partners, and internal stakeholders.
• Develop and tune custom kernels and operator-level optimizations to maximize throughput and minimize latency.
• Drive quantization, sparsity, and batching strategies tailored to D-Matrix computational model.
• Build and maintain inference runtimes, serving frameworks, and evaluation tooling.
• Contribute to distributed inference systems: tensor/pipeline parallelism, disaggregated prefill/decode, KV-cache management.
• Work closely with hardware architects to provide firmware and compiler teams with actionable inference workload insights.
• Partner with product and business development to translate POCs into customer-facing demonstrations.
• Contribute to technical publications, whitepapers, and open-source projects that advance D-Matrix visibility.

Principal LLM Inference Engineer

• Strategic Account Ownership: Own and grow strategic customer relationships, acting as the primary point of contact for executive stakeholders and technical teams alike.
• Drive AI Infrastructure Revenue: Accelerate the adoption of Vultr’s AI cloud infrastructure solutions by identifying opportunities and guiding customers through the sales cycle—from initial engagement through solution architecture and scale-up.
• Customer-Centric Engagement: Understand customer priorities, technical requirements, and business goals to position Vultr’s value proposition effectively and deliver tailored solutions.
• Trusted Advisor: Provide thought leadership on AI/ML trends, infrastructure needs, and optimization strategies to senior leaders within your accounts. Help customers navigate the AI landscape and make informed architectural decisions.
• Collaborate for Success: Work cross-functionally with Product Management, Solutions Engineering, and Customer Success to ensure alignment on product capabilities, roadmap feedback, and long-term success.
• Sales Process Excellence & Operational Hygiene: Document and maintain accurate sales activity data in CRM tools to support MRR forecast accuracy.
• AI Ecosystem Engagement: Understand and leverage the AI partner ecosystem to enhance Vultr’s value proposition for Founders and C-suite executives.

Account Executive, AI Infrastructure Sales

• Design and maintain a safety evaluation framework—adversarial prompt sets, scenario-based test suites, and regression suites—so that every model and agent update is validated before it ships.
• Lead structured red-teaming exercises covering jailbreaks, prompt injection, tool misuse, and data exfiltration; document findings and drive each issue through to remediation and closure.
• Build and iterate on guardrail logic, including input/output filtering, tool-boundary constraints, action validation, sensitive-data redaction, and policy prompting.
• Integrate safety checks into CI/CD and runtime so that unsafe behavior is intercepted before it reaches users.
• Perform threat modeling for agentic scenarios: tool-call boundaries, sandbox isolation, and least-privilege access, with particular attention to preventing agents from exfiltrating data or executing irreversible actions through chained tool calls.
• Conduct safety reviews of reinforcement-learning (RL) environments and trajectory data, partnering with environment and agent engineering teams to embed safety constraints directly into the environments themselves.
• Instrument AI features for safety with structured logging, tracing, and metrics, enabling detection of unsafe patterns and regressions in production.
• Prepare evidence for governance reviews—test reports, evaluation summaries, and mitigation validation—aligned with internal Responsible AI standards.
• Collaborate with Product and UX to improve safety interactions (warnings, confirmations, refusal messaging, and feedback collection), and align evaluation goals with the Research and Data teams.

AI/LLM Safety Engineer

• Build and maintain agentic systems that continuously collect, process, and surface signals from social platforms, on-chain data, and ads ecosystems. Turn raw, multi-source data into structured intelligence that informs real account and campaign decisions.
• Design and maintain the pipelines that power those systems: ingesting from multiple external APIs, enriching and modeling data, and building semantic layers that both humans and AI agents can actually use.
• Build multi-step AI agents that automate research, synthesis, monitoring, and reporting. Not single-call LLM wrappers, but orchestrated systems with memory, defined tools, and structured output schemas that teams can depend on.
• Build full-stack applications and dashboards that make intelligence accessible across client accounts. Live tools, not decks.
• Spend real time with internal teams across marketing, PR, content, growth, and social before writing a line of code. Map how work actually moves, identify where signal gaps create friction, and build systems that solve for multiple teams from a single architecture.
• Stay ahead of the AI tooling landscape. Know when a no-code connector is the right call and when you need to build. Define how impact gets measured, and build the reporting to prove it.

AI Infra Engineer

• Design, build, and maintain scalable ML infrastructure and pipelines supporting model training, deployment, monitoring, governance, and lifecycle management.
• Develop and optimize CI/CD pipelines for machine learning and AI workloads across development, staging, and production environments.
• Build reusable ML platform capabilities including feature stores, model registries, experimentation frameworks, artifact management, and deployment automation.
• Implement scalable orchestration and workflow solutions for batch and real-time ML inference workloads.
• Create robust monitoring systems to measure model performance, detect model drift, monitor data quality, and ensure production reliability.
• Develop automation tools and self-service capabilities to improve the efficiency, scalability, and reliability of MLOps processes.
• Collaborate with Data Scientists and Software Engineers to streamline the ML lifecycle from experimentation through enterprise production deployment.
• Apply software engineering best practices to AI/ML systems including testing, observability, resiliency, security, versioning, and infrastructure-as-code.
• Identify gaps and improvement opportunities within the organization’s ML platform ecosystem and architect scalable solutions to address them.
• Support enterprise AI governance, compliance, auditability, and model risk management requirements.
• Ensure platform scalability, reliability, security, and operational excellence across AI/ML systems.
• Lead the architecture, design, and deployment of enterprise Generative AI solutions leveraging LLMs, foundation models, and agentic AI systems.
• Design and implement Retrieval-Augmented Generation (RAG) pipelines using vector databases, embeddings, semantic search, reranking, and retrieval optimization strategies.
• Build scalable LLM orchestration frameworks using technologies such as LangChain, LlamaIndex, Semantic Kernel, or equivalent frameworks.
• Develop advanced prompt engineering strategies, prompt chaining, context management, and agent workflows to improve LLM accuracy and reliability.
• Evaluate and implement fine-tuning, parameter-efficient tuning, and prompt-based optimization approaches for domain-specific use cases.
• Build AI evaluation and benchmarking frameworks to measure hallucination rates, response quality, grounding accuracy, toxicity, bias, latency, and business performance metrics.
• Implement AI safety guardrails, governance controls, content filtering, and responsible AI practices for enterprise healthcare environments.
• Design scalable GenAI APIs and microservices supporting high-throughput enterprise AI applications.
• Optimize GenAI systems for cost, latency, throughput, and inference performance across cloud and hybrid environments.
• Integrate enterprise data sources, healthcare systems, and knowledge repositories into secure GenAI workflows.
• Research and evaluate emerging GenAI technologies, open-source frameworks, and foundation models to drive innovation and continuous improvement.
• Develop architecture diagrams, technical roadmaps, implementation strategies, and executive-level documentation for enterprise AI initiatives.
• Collaborate with cybersecurity, compliance, and infrastructure teams to ensure secure and compliant deployment of GenAI solutions involving PHI and sensitive healthcare data.
• Contribute to the development of AI platform standards, reusable GenAI accelerators, templates, and engineering best practices.

Senior MLOps, Generative AI Engineer

• Design and develop AI applications using Amazon Bedrock
• Build and configure Bedrock Agents and Action Groups
• Implement Retrieval-Augmented Generation (RAG) architectures
• Develop natural-language-to-API workflows
• Configure Bedrock Knowledge Bases and vector search solutions
• Implement AI safety controls, content filtering, and PII protection
• Build and maintain serverless AWS solutions
• Deploy infrastructure using Infrastructure-as-Code practices
• Monitor AI system performance, usage, and operational metrics

AI Engineer, Generative AI

• Strategic Account Ownership
• Drive AI Infrastructure Revenue
• Customer-Centric Engagement
• Trusted Advisor
• Collaborate for Success
• Sales Process Excellence & Operational Hygiene
• AI Ecosystem Engagement

Senior Account Executive, AI Infrastructure Sales

• Define the resilience platform, own the product roadmap and delivery for specific platform features
• Collaborate with the open-source developer community — prioritizing GitHub issues, gathering feedback, supporting contributors, and channeling community signal back into the roadmap
• Collaborate with engineering on feature design, prioritization, execution, and architecture tradeoffs
• Align cross-functionally with other Product, Engineering, Product Marketing, and Field teams on requirements, roadmaps, messaging, and engagements

Technical Product Manager, AI Infrastructure Resilience

• lead the product marketing strategy for Cisco’s AI infrastructure portfolio
• connect technical capabilities to the business outcomes customers require for AI workloads and AI-native applications
• define the narrative, positioning, messaging, launch strategy, and sales enablement required to power secure, connected, AI-ready organizations
• partner with Product Management, Engineering, Sales, Corporate Marketing, Analyst Relations, and Communications
• translate sophisticated infrastructure innovation into market value
• drive cross-functional alignment
• equip the business to win with enterprise buyers and executives

Director of Product Marketing Strategy – AI Infrastructure

• Own and drive the global 4P strategy for ACS AI infrastructure platforms, from annual planning through execution and performance tracking.
• Define clear value propositions, positioning, and segmentation for AI infrastructure solutions aligned to customer workloads and industry use cases.
• Drive execution of the AI GTM plan with regions and countries, ensuring timely rollout and consistency.
• Act as the WW GTM expert for ACS AI infrastructure solutions (e.g., ZGX Nano, ZGX Fury, Boost), ensuring tight alignment between product capabilities and market needs.
• Translate complex AI infrastructure architectures into compelling, sales‑ready GTM narratives and offers.
• Partner with GBU and engineering teams to influence roadmap priorities based on market and customer feedback.
• Support priority deals, lighthouse customers, and strategic pursuits requiring AI infrastructure expertise.
• Enable sales teams with clear guidance, playbooks, and tools to accelerate AI‑led pipeline generation and conversion.
• Collaborate closely with alliance teams (e.g., NVIDIA, AMD, Intel, ISVs) to deliver joint GTM programs, aligned launches, and co‑selling motions tied to AI infrastructure platforms.
• Define and track KPIs to measure GTM effectiveness, pipeline impact, and revenue outcomes.

LLM Engineer jobs in United States

See all jobs on JobTailor