Staff Machine Learning Engineer
Fullscript
full-time
Posted on:
Location Type: Remote
Location: Canada
Visit company websiteExplore more
Job Level
About the role
- Lead the design, development, and deployment of production, multi-turn LLM-powered features, including summarization tools and clinician-facing conversational agents that support follow-up questions and reasoning over clinical context
- Own backend services in Python that integrate LLM agents with Fullscript’s platform and support reliable production use
- Help define technical direction for prompting, grounding, safety, and orchestration strategies used across clinical AI workflows
- Establish and improve evaluation approaches for LLM outputs, including accuracy, hallucinations, edge cases, and overall feature quality
- Shape engineering patterns for model-related workflows, including testing, CI/CD, observability, and version control
- Partner with medical, product, and engineering teams to identify high-value opportunities for AI and turn them into practical, scalable product capabilities
- Work cross-functionally with engineering, analytics, and medical SMEs to refine requirements and ensure data and system design support clinical use cases
- Provide technical leadership across projects by creating clarity in ambiguous problem spaces, guiding tradeoff decisions, and raising the quality bar for the team
- Stay current with the latest LLM research and emerging AI technologies, and help assess where they can be applied effectively at Fullscript
Requirements
- 6+ years of experience building and implementing machine learning applications in production, including meaningful experience with LLM-powered agents, conversational experiences, or agent-based workflows
- A track record of owning complex technical problems end to end and shaping implementation beyond your immediate code contributions
- Experience designing and deploying AI systems that answer open-ended questions, support follow-up interactions, and operate reliably in production
- Strong experience with LLM application frameworks and tooling, such as LangChain, LangGraph, or similar orchestration and RAG frameworks
- Familiarity with evaluation and monitoring frameworks for LLM outputs, conversational quality, and system reliability
- Knowledge of MCP, agent orchestration patterns, or related approaches for building multi-step AI systems
- Strong proficiency in Python and SQL
- Experience making sound technical decisions around quality, safety, maintainability, and scalability in production AI systems
- Strong communication and collaboration skills, with the ability to work effectively across technical and non-technical stakeholders.
Benefits
- Flexible PTO & competitive pay—rest fuels performance.
- RRSP match & stock options—invest in your future.
- Customizable benefits—flexible coverage, paramedical services, and an HSA.
- Fullscript discounts—save on wellness products.
- Continuous learning—training budget + company-wide initiatives.
- Wherever You Work Well—hybrid and remote flexibility.
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard Skills & Tools
machine learningLLM-powered agentsconversational experiencesAI systems designPythonSQLevaluation frameworksmonitoring frameworksagent orchestration patternsCI/CD
Soft Skills
technical leadershipcommunicationcollaborationproblem-solvingdecision-making