Pieces

AI Research Scientist

• Develop the AI evaluation modules under the large language model operations (LLMOps), which includes the components of LLM observability, benchmark datasets, standardization of metrics, evaluation of complex LLM workflow, knowledge graph, etc
• Implement strategies of AI safety for adversarial detection AI, comprehensive model evaluation, and error analysis, incorporating human feedback for continuous AI model improvement
• Maintain and continuously improve existing data models and AI Engine pipelines, including monitoring performance, resolving production issues, optimizing for scalability, and ensuring robust dataflows across the AI/ML and application stack.
• Apply LLM observability tools, such as LangFuse, Arize Phoenix, Braintrust, etc, for LLM traces, latency and cost awareness, evaluation, and prompt management/deployment.
• Develop data pipelines in AI engine to support a generic data store and data representation, including a meta-data layer for clinical semantics and a vector database for downstream applications
• Develop generative and predictive AI modules and necessary data transformation modules in AI engine alongside agentic AI systems
• Support agentic AI systems development using LangGraph and/or other agentic AI frameworks for the use cases such as AI assistants in both internal applications (e.g. SafeRead) and external ones (e.g. SmarterNotes)
• Collaborate with clinical domain experts and product management teams to translate clinical guidelines into AI modules, ensuring that the developed products align with clinical needs and user requirements.
• Stay updated with the latest AI advancements in the generative AI era, integrating new knowledge into practical applications and contributing to the AI community through professional engagements.

Applied AI Scientist