Senior Data Scientist
McKesson
CloudPythonSQL
Senior Data Scientist, ML Operations and Analytics
Filevine
contract
Posted on:
Origin: • 🇨🇿 Czech
Visit company websiteJob Level
Senior
Tech Stack
CloudPythonSQL
About the role
- Build ML operations analytics tracking pipeline health, costs, latency, and quality metrics across all services
- Lead LLM evaluation initiatives using frameworks like Braintrust or Langfuse, designing custom metrics, quality monitoring and LLM observability
- Collaborate with our data annotation team on prompt engineering and evaluation dataset creation
- Aggregate and operationalize user feedback for model improvement
- Create actionable dashboards and insights for ML teams and leadership
- Help define SLAs, cost optimization and failure prevention strategies for ML pipelines
- Contribute to building analytics layer ensuring operational excellence and continuous improvement across AI systems
- Participate in cross-functional collaboration with ML, engineering, product, legal, and operations teams
Requirements
- 3+ years in data science, ML operations, or analytics engineering
- Experience with LLM evaluation frameworks and quality metrics design
- Strong Python, SQL, and data visualization skills
- Knowledge of prompt optimization techniques
- Excellent communication skills in English to translate technical metrics into business insights
- Collaborative mindset with ability to work across machine learning teams
- Ability to meet periodically in our Prague office as we value in-person collaboration
