Manager

• Lead the development and execution of NCBE’s grading and scoring strategy across all item types, including multiple-choice and constructed-response components of the NextGen UBE
• Coordinate cross-functionally among test development, psychometrics, test operations, product management, and technology teams to ensure alignment and efficient delivery of scoring workflows and policies
• Define and maintain the technical roadmap for grading and scoring, detailing system enhancements, automation initiatives, and integrations with delivery and scoring platforms
• Oversee operational readiness for scoring activities, ensuring that all scoring systems, processes, and personnel are prepared for each administration
• Partner with psychometricians and data scientists to ensure scoring models and methods meet NCBE’s validity, reliability, and fairness standards
• Support research and development in automated scoring, including model evaluation, calibration, and pilot testing, and ensure responsible AI use consistent with NCBE’s principles
• Collaborate with vendor partners (e.g., scoring platforms, AI vendors, and analytics providers) to ensure seamless integration, technical quality, and adherence to NCBE’s security and compliance standards
• Monitor scoring quality and performance metrics, such as rater reliability, automation accuracy, turnaround time, and system uptime, and recommend continuous improvements
• Document and communicate product requirements and functional specifications to engineering, psychometric, and operational teams
• Ensure data governance, security, and compliance for all grading and scoring systems, consistent with NCBE policies and legal/licensure requirements
• Prepare reports and presentations summarizing grading performance, research outcomes, and roadmap progress for internal stakeholders, governance committees, and the NCBE Board.

Scoring and Grading Capability Manager

Care Manager, RN

B2B MarTech Manager

PMO Manager

Senior Manager, Incident Management

Manager, Practice Solutions