Senior Generative AI Scientist II – Model Risk & Validation

Cotiviti

Senior GenAI Scientist II delivering AI solutions to reduce healthcare costs and improve outcomes. Focused on model risk management, validation, metrics, and AI ethics.

Posted 6/23/2026full-timeRemote • 🇺🇸 United StatesSenior💰 $145,000 - $170,000 per yearWebsite

ATS Keywords

Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills

model validationmodel risk managementAI/ML model developmentnatural language processingtransformersmachine learningdata scienceResponsible AImodel evaluationproduction-grade machine learning deployments

Soft Skills

communicationcollaborationproblem-solvingadaptabilitycritical thinkingattention to detailcreativityleadershipinterpersonal skillslearn it all mindset

Tools & Technologies

HuggingFaceLangchainLLAMAMistralOpenAIpandasscikit-learnkerasTensorFlowPyTorch

Industry Keywords

model driftdata driftmodel monitoringexplainabilityAI risk management frameworksbias and fairness evaluationmodel metricsAWSAzureGCP

Tech Stack

Tools & technologies

ApacheAWSAzureGoogle Cloud PlatformKerasPandasPyTorchScikit-LearnSparkTensorflow

About the role

Key responsibilities & impact

delivering solutions that help our clients identify payment integrity issues, reduce the cost of healthcare processes, or improve the quality of healthcare outcomes.
conduct independent model validation of existing models for benchmarking, assessment, and gauging effectiveness.
determine aspects of model drift and related data drift for the purpose of model risk management (MRM).
apply deep expertise with AI/ML/GenAI model development, including hands-on experience with model building and model evaluation.
benchmark and potentially rebuild existing models as needed using updated data, and potentially newer, more modern and effective algorithms.
actively drive improvements in model monitoring activities, including methods for model registration, model metadata management, and conceptualizing approaches for related tools and techniques.
complete all responsibilities as outlined in the annual performance review and/or goal setting.
complete all special projects and other duties as assigned.

Requirements

What you’ll need

Graduate Degree in a quantitative discipline such as Computer Science/Engineering, Statistics, Operations Research covering Advanced Statistics, Machine learning and AI.
Experience with the latest techniques in natural language processing including transformers, fine-tuning LLMs, measuring/benchmarking and deploying LLMs with tools such as HuggingFace, Langchain, LLAMA/Mistral and OpenAI, vector databases.
5+ years of hands-on data science/AI experience, using typical machine learning and data science tools including pandas, scikit-learn, keras, nltk, and TensorFlow/PyTorch, GPU.
General understanding of Responsible AI (RAI), including explainability (XAI), AI NIST RMF, and related AI risk management frameworks.
Experience and understanding evaluating models for bias and fairness, with aptitude for detecting bias in the model design and data, as well as using metrics such as SHAP and LIME.
Understanding appropriate model metrics and techniques for managing, evaluating and monitoring GenAI models and LLMs
Experience building production-grade machine learning deployments on AWS, Azure, or GCP.
Experience working with Apache Spark™ and large-scale distributed datasets.
Experience communicating technical concepts to non-technical and technical audiences is a plus.
Passion for collaboration, learn it all mindset and driving value with AI.

Benefits

Comp & perks

medical, dental, vision, disability, and life insurance coverage
401(k) savings plans
paid family leave
9 paid holidays per year
17-27 days of Paid Time Off (PTO) per year, depending on specific level and length of service with Cotiviti