Data Scientist

AnswersNow

Data Scientist optimizing AI systems for autism therapy at AnswersNow. Lead design and evaluation of AI-driven clinical recommendations using real-time clinical data.

Posted 7/6/2026full-timeRemote • 🇺🇸 United StatesMid-LevelSenior💰 $144,000 - $168,000 per yearWebsite

Tech Stack

Tools & technologies

AssemblyPython

About the role

Key responsibilities & impact

Architect and continuously improve the RAG pipeline that retrieves client-specific clinical context — session notes, treatment plan goals, historical performance data — and injects it into inference-time prompts
Design the retrieval layer: chunking strategies, embedding models, vector store configuration, and retrieval ranking — optimizing for clinical relevance, not just semantic similarity
Build a context assembly system that selects and structures the most relevant clinical information for each model invocation, given token constraints and clinical priority
Evaluate retrieval quality rigorously: build test sets, measure recall and precision, and iterate on the pipeline based on where retrieval fails
Design evaluation frameworks that assess AI recommendation quality beyond standard NLP metrics — working with clinical stakeholders to define what 'good' means for each use case
Build automated evaluation pipelines that can test AI outputs at scale: LLM-as-judge evaluators, human review workflows, and clinical validity checks
Maintain evaluation datasets that reflect the real distribution of clinical scenarios the model encounters in production
Systematically identify where foundation model capabilities fall short for AnswersNow's care model: what clinical reasoning the model gets wrong, what it hallucinates, what it doesn't know how to handle
For each identified gap, recommend and implement the appropriate mitigation — improved retrieval, prompt engineering, output validation, or escalation to human review
Monitor production AI outputs for quality, drift, and failure modes using the evaluation infrastructure you've built
Define alerting thresholds and escalation paths for when AI quality falls below acceptable clinical standards
Work closely with clinical leadership and BCBAs to understand the care model deeply enough to design AI systems that support it accurately
Translate clinical domain knowledge into technical requirements: what context does the model need, what outputs are clinically acceptable, where does the model need to defer to the clinician

Requirements

What you’ll need

4+ years of experience in applied data science, ML engineering, or AI engineering in a production environment
Deep understanding of RAG architectures: retrieval systems, embedding models, vector databases (Pinecone, Weaviate, pgvector, or similar), chunking strategies, and context assembly
Experience designing and running evaluation frameworks for AI systems — you've thought hard about how to measure quality in domains where ground truth is ambiguous
Strong Python skills; experience with LLM orchestration frameworks (LangChain, LlamaIndex, or similar)
Clinical NLP experience or healthcare AI background is strongly preferred — you understand why clinical data is different from general text and what that means for AI system design
You think like an engineer and a scientist: you build systems that can be measured, iterated on, and trusted — not black boxes
Strong written communication: you can explain RAG pipeline design to a clinician and explain clinical requirements to an engineer
Genuine interest in the clinical domain — you want to understand Applied Behavior Analysis well enough to build AI that actually helps BCBAs do their jobs

Benefits

Comp & perks

Fully remote – work from anywhere in the U.S.
Flexible hours with an async-friendly team culture

ATS Keywords

✓ Tailor your resume

Applicant Tracking System Keywords

Tip: use these terms in your resume and cover letter to boost ATS matches.

Hard Skills & Tools

Data ScienceMachine Learning EngineeringAI EngineeringEmbedding ModelsVector DatabasesChunking StrategiesContext AssemblyQuality MeasurementAutomated Evaluation PipelinesPrompt Engineering

Soft Skills

Strong Written CommunicationInterdisciplinary Collaboration