AI Engineer III

CareSource

AI Engineer III responsible for building reliable AI infrastructures and deploying AI solutions at CareSource. Leading initiatives in LLMOps and Azure AI Foundry stack for efficient operations.

Posted 4/11/2026full-timeRemote • 🇺🇸 United StatesMid-LevelSenior💰 $94,100 - $164,800 per yearWebsite

Tech Stack

Tools & technologies

AzureCloudKubernetesPythonTerraform

About the role

Key responsibilities & impact

Architect and maintain the LLMOps/GenAIOps toolchain, including model registries, prompt version control, and reproducible training pipelines.
Implement and manage the Azure AI Foundry environment, configuring model routers, quota management, and private endpoints for secure inferencing.
Develop comprehensive observability dashboards to track model latency, token costs, hallucination rates, and drift.
Automate "Policy-as-API" controls within the orchestration layer to enforce governance guardrails (e.g., PII filtering) at runtime.
Collaborate with the Platform SRE team to ensure high availability and disaster recovery for mission-critical clinical agents.
Manage the "Model Registry," ensuring all deployed models have associated version history, performance metrics, and rollback targets.
Configure and maintain "Vector Databases" and RAG pipelines, optimizing retrieval performance and index freshness.
Implement "Prompt Filtering" and content moderation gateways to prevent jailbreaks and enforce safety standards at the infrastructure level.
Develop "Blue/Green" or "Canary" deployment strategies for AI agents to safely test new model versions in production.
Manage the "API Gateway" for all AI services, ensuring authentication, rate limiting, and usage logging are enforced.
Optimize "GPU/CPU Orchestration" to control compute costs while maintaining performance SLAs for high-volume inference.
Build automated "Drift Detection" alerts that trigger retraining or human review when model performance degrades below a set threshold.
Perform any other job related duties as requested.

Requirements

What you’ll need

Bachelor's degree in Computer Science, Engineering, or related technical field required
Five (5) years of IT engineering experience, with at least three (3) years specialized in DevOps, MLOps, or Cloud Infrastructure required
Experience with Azure AI Services (Azure OpenAI, AI Search, Azure ML) and container orchestration (Kubernetes/AKS) required
Experience building and maintaining CI/CD pipelines for machine learning models or complex software applications required
Mastery of Python and scripting languages for automation and infrastructure-as-code (Terraform, Bicep, ARM templates)
Deep understanding of LLMOps principles: Prompt versioning, model registry management, and evaluation pipelines (e.g., MLflow, Prompt Flow)
Proficiency in Azure Networking and Security, including Private Endpoints, VNET integration, and API Management (APIM) configuration
Knowledge of Vector Databases and RAG (Retrieval Augmented Generation) infrastructure requirements
Strong observability skills, utilizing tools like Azure Monitor or App Insights to track token usage, latency, and drift

Benefits

Comp & perks

Comprehensive total rewards package
Health insurance
Paid time off
Flexible working arrangements
Professional development opportunities

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

PythonTerraformBicepARM templatesCI/CD pipelinesMLOpsDevOpsAzure AI ServicesKubernetesLLMOps

Soft Skills

collaborationobservabilitygovernanceautomationproblem-solvingcommunicationorganizational skillsdisaster recoveryhigh availabilityperformance optimization

Certifications

Bachelor's degree in Computer ScienceBachelor's degree in Engineeringrelated technical field