AI Infrastructure Engineer – Agents, ML Systems

HavocAI

AI Infrastructure Engineer building secure LLM agents, RAG pipelines, and ML tooling. Supporting defense autonomy teams with reliable internal AI systems and workflows.

Posted 8/5/2026full-timeRemote • 🇺🇸 United StatesMid-LevelSenior💰 $175,000 - $200,000 per yearWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in building and maintaining AI infrastructure, integrating various data sources and tools, and ensuring the security and reliability of agentic AI systems. Proficient in programming languages and familiar with ML workflows, data pipelines, and observability practices.

Highest-signal resume keywords

Python ProgrammingML Infrastructure SupportAI Tool IntegrationData Pipeline DevelopmentSecurity Clearance

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

TypeScript ProgrammingGo ProgrammingC++ ProgrammingAPI DevelopmentData EngineeringMLOpsModel EvaluationFine-TuningDebugging Complex SystemsEmbedding Management

Soft Skills

Collaboration Across TeamsProblem-Solving

Tools & Technologies

KubernetesDockerRayAirflowDagsterMLflowWeights & BiasesKafkaPostgresS3-Compatible Storage

Certifications & Qualifications

Active Security Clearance

Industry Keywords

Agentic AI SystemsRAG PipelinesTelemetrySimulationDefense Technology

Tech Stack

Tools & technologies

AirflowDistributed SystemsDockerGoKafkaKubernetesPostgresPythonRayTypeScript

About the role

Key responsibilities & impact

Build internal AI infrastructure connecting LLMs and AI agents with internal tools, APIs, data sources, data lakes, telemetry stores, simulation tools, code repositories, documentation systems, logs, and engineering workflows.
Develop and maintain agentic AI systems for task automation, data analysis, engineering support, simulation workflows, and internal productivity.
Build tool integration and connector infrastructure for AI agents, including MCP and other emerging tool-use standards.
Create pipelines for retrieval, RAG, context management, document processing, embeddings, and internal knowledge search.
Support ML infrastructure workflows including data preparation, dataset curation, experiment tracking, model evaluation, fine-tuning support, and model deployment.
Build evaluation frameworks for agent performance, tool-use reliability, task success, model quality, regression testing, and failure analysis.
Develop observability, logging, tracing, auditability, monitoring, and debugging tools for AI agents, model calls, MCP tools, and ML pipelines.
Partner with Autonomy, Software, Data, Simulation, Product, and Operations teams to identify AI use cases and build reliable internal tools.
Secure agentic AI systems using least-privilege access, sandboxed execution, prompt-injection mitigation, secrets management, human approvals, and safe handling of sensitive and defense data.
Maintain documentation, reusable examples, templates, and best practices for safe AI-tool adoption.

Requirements

What you’ll need

Bachelor's degree in Computer Science, Engineering, Machine Learning, Data Science, Applied Mathematics, or a related technical field.
3+ years of experience in software engineering, infrastructure engineering, ML infrastructure, backend systems, data engineering, developer tools, or related technical roles.
Strong programming experience in Python, TypeScript, Go, C++, or similar languages.
Experience building production software systems, APIs, services, data pipelines, or internal platforms.
Experience with, or strong interest in, LLM applications, AI agents, tool-using systems, RAG pipelines, or AI developer tools.
Familiarity with embeddings, vector search, prompt management, evaluation, model serving, fine-tuning, or MLOps.
Ability to integrate APIs, databases, object stores, documents, logs, internal tools, and structured or unstructured data sources.
Strong understanding of reliability, observability, testing, and maintainability.
Strong grounding in securing AI and agentic systems, including least-privilege tool access, prompt-injection and misuse mitigation, secrets management, and safe handling of sensitive data.
Ability to work across Software, Data, ML, Infrastructure, and Product teams.
Strong debugging skills and comfort working with complex distributed systems.
U.S. citizenship and ability to obtain and maintain a security clearance.
Preferred: experience with MCP, AI tooling, vector databases, embeddings, retrieval, RAG, fine-tuning, Kubernetes, Docker, Ray, Airflow, Dagster, MLflow, Weights & Biases, Kafka, Postgres, S3-compatible storage, internal platforms, workflow automation, human-in-the-loop systems, engineering integrations, autonomy, robotics, simulation, telemetry, perception, defense technology, or secure AI systems.
Active or prior security clearance preferred.

Benefits

Comp & perks

100% Employer paid Health, Dental and Vision Insurance for you and your families
Life Insurance (Employer Paid)
Ability to participate in the companies 401k program (Matching)
Unlimited PTO policy with an enforced 2 week minimum
Equity Package
Work / Home Office Stipend
Global Entry
16 Week Paid Parental Leave
Monthly Health and Wellness Stipend
Bonus offered (compensation summary)
Remote work arrangement