Senior AI Engineer

Employer Direct Healthcare

Senior AI Engineer building production GenAI, RAG, and agentic systems for Lantern’s specialty healthcare platform. Leading AI architecture, LLMOps, mentoring, and cross-functional delivery in regulated healthcare.

Posted 8/4/2026full-timeNew York City • New York, Texas • 🇺🇸 United StatesSeniorWebsite

Core Competencies

Role fit

Core Competencies

Use this summary to align your resume positioning with the role.

Demonstrates expertise in architecting and delivering LLM-powered capabilities, including prompt engineering, multi-agent orchestration, and production AI/ML systems. Proficient in Python, LLM APIs, and cloud platforms, with a strong focus on quality benchmarks and observability standards.

Highest-signal resume keywords

LLM Architecture and ImplementationPrompt Engineering StrategyProduction AI/ML Systems DeploymentCloud Platforms (Azure)Mentoring and Technical Leadership

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

PythonLLM APIs (OpenAI, Azure OpenAI, Anthropic, Google)RAG Systems Design and DeploymentAgentic Frameworks (LangChain, LangGraph, AutoGen)LLMOps/MLOps Tooling (MLflow, Azure ML, Weights & Biases)Containerization (Docker, Kubernetes)Multi-Step Agent WorkflowsLLM Evaluation Tooling (RAGAS, TruLens, DeepEval)Production Code Quality StandardsVersion Control

Soft Skills

Strong CommunicationCollaborationTechnical Discussion LeadershipMentoring

Tools & Technologies

Azure Data PipelinesVector DatabasesHybrid Retrieval ArchitecturesMonitoring and Observability ToolsAudit Logging Systems

Certifications & Qualifications

Bachelor’s or Master’s Degree in Computer Science or Engineering

Industry Keywords

GenAIAI SafetyCompliance ControlsHealthcare DataReal-Time Inference

Tech Stack

Tools & technologies

AzureCloudDockerKubernetesPython

About the role

Key responsibilities & impact

Architect and deliver production LLM-powered capabilities, including advanced RAG pipelines, structured extraction, multi-document reasoning, dialogue systems, and domain-specific language models
Own prompt engineering strategy and establish standards for prompt management, evaluation, and continuous improvement
Select and integrate embedding models, vector databases, and hybrid retrieval architectures
Define LLM evaluation frameworks, quality benchmarks, guardrails, grounding strategies, and hallucination mitigation controls
Evaluate frontier and open-source models and lead model selection decisions
Lead architecture and implementation of production agentic systems with multi-agent orchestration, planning, tool use, memory, and state persistence
Design human-in-the-loop mechanisms, approval workflows, fallback strategies, and audit trails
Establish reliable tool-use and function-calling patterns for external APIs, clinical systems, and internal data services
Define agent observability standards, including trace logging, monitoring, drift detection, and structured evaluation
Write production-quality code and set standards through design and code reviews
Architect and maintain LLM inference services, API integrations, and Azure data pipelines
Define and champion LLMOps practices, including prompt versioning, experiment tracking, model registration, A/B testing, CI/CD, and regression testing
Establish production monitoring and observability for latency, quality, cost, safety, and behavioral drift
Lead technical documentation, including architecture decision records, runbooks, model cards, and evaluation playbooks
Partner with product, clinical operations, marketing, and data teams to translate requirements into AI initiatives
Mentor junior and mid-level engineers
Lead architecture discussions and contribute to the AI engineering roadmap
Drive organizational adoption of GenAI and agentic capabilities

Requirements

What you’ll need

Bachelor’s or Master’s degree in Computer Science, Engineering, or equivalent practical experience
5+ years of experience building and deploying production AI/ML systems, with at least 2–3 years focused on LLM and GenAI applications
Strong proficiency in Python and software engineering fundamentals, including testing, modular design, code reviews, documentation, and version control
Deep hands-on experience with LLM APIs such as OpenAI, Azure OpenAI, Anthropic, and Google
Advanced prompt engineering experience, including chain-of-thought, few-shot, structured outputs, tool-calling, and multi-turn dialogue
Proven experience designing and deploying production RAG systems, including document processing, chunking, embeddings, vector databases, hybrid retrieval, and retrieval evaluation
Hands-on experience with agentic frameworks such as LangChain, LangGraph, AutoGen, CrewAI, or custom orchestration
Experience deploying multi-step agent workflows in production
Experience with LLM evaluation tooling such as RAGAS, TruLens, DeepEval, or custom frameworks
Experience with cloud platforms, preferably Azure, and containerized deployment using Docker and Kubernetes
Familiarity with LLMOps/MLOps tooling such as MLflow, Azure ML, and Weights & Biases
Strong communication and collaboration skills; ability to lead technical discussions and influence cross-functional partners
Track record of mentoring engineers and elevating engineering standards
Preferred additional experience with regulated-industry AI safety, grounding, compliance controls, audit logging, PII handling, fine-tuning, RLHF, DPO, LoRA, QLoRA, healthcare data, multi-agent systems at scale, streaming pipelines, real-time inference, event-driven architectures, open-source contributions, research publications, or conference presentations

Benefits

Comp & perks

Medical Insurance
Dental Insurance
Vision Insurance
Short & Long Term Disability
Life Insurance
401k with company match
Paid Time Off
Paid Parental Leave