Reinforcement Learning Research Intern – Game AI
Xibo Open Source Digital Signage
Python
Generative AI Post-Doctorate Fellow
The College Board
full-time
Posted on:
Location Type: Remote
Location: Remote • 🇺🇸 United States
Visit company websiteSalary
💰 $104,000 - $170,000 per year
Job Level
Mid-LevelSenior
Tech Stack
PythonSQL
About the role
- Develop best practices and design principles for new AI-driven learning & assessment prototypes (33%)
- Survey and synthesize the research base on adaptive learning, gamification, and non-cognitive assessment to define evidence-backed design heuristics.
- Lead co-design sprints with psychometricians, UX researchers, and engineers to embed those heuristics in prototypes and pilots.
- Build evaluation frameworks (validity, reliability, engagement, fairness) to judge prototype performance and iterate rapidly.
- Author “Design Principles” playbooks and contribute to internal standards that guide future College Board AI initiatives.
- Distill learning-science and assessment best practices into effective LLM contexts (33%)
- Translate analytic rubrics, exemplar responses, and scoring anchors into high-leverage prompt libraries, system messages, and retrieval chains for LLM-based tutors and scorers.
- Prototype few-shot and chain-of-thought prompting strategies that surface metacognitive feedback and scaffold student reasoning.
- Collaborate with engineers to automate rubric-to-prompt conversions and ensure version control across iterative assessment designs.
- Establish validation pipelines—quantitative (e.g., QWK, IRT metrics) and qualitative (e.g., think-aloud reviews)—to monitor alignment between LLM outputs and intended learning goals.
- Supporting LLM prototype experiments informed by learning science/assessment best practices (33%)
- Co-design and analyze rapid LLM prototype experiments (A/B & quasi-experimental) to validate pedagogical soundness, fairness, and psychometric quality.
Requirements
- PhD (completed by Aug 2025) in Learning Sciences, Educational Psychology, AIED, Measurement, Cognitive Science, or related field
- Demonstrated research on adaptive/gamified or non-cognitive assessment design
- Hands-on experience with large-language-model prompting / evaluation (OpenAI, Anthropic, etc.) or learning-analytics pipelines.
- Strong expertise in statistical modeling and data analysis using appropriate statistical methods and software (e.g., SAS, Stata, SPSS, R, Python); familiarity with IRT or ML fairness a plus.
- Experience with large, national datasets related to the education industry and the ability to work accurately and quickly with new datasets
- Experience using SQL to access data for data analysis and/or a willingness to learn
- Experience with survey design and analysis, and Qualtrics a plus
- Adept problem-solving skills, including using data to inform decisions and actions
- The ability to travel 4-6 times a year to College Board offices or on behalf of College Board business
- Authorization to work in the United States
Benefits
- Annual bonuses and opportunities for merit-based raises and promotions
- A mission-driven workplace where your impact matters
- A team that invests in your development and success
Applicant Tracking System Keywords
Tip: use these terms in your resume and cover letter to boost ATS matches.
Hard skills
AI-driven learning designadaptive learninggamificationnon-cognitive assessmentstatistical modelingdata analysislarge-language-model promptingSQLsurvey designevaluation frameworks
Soft skills
problem-solvingcollaborationcommunicationleadershiporganizational skillsanalytical thinkingcreativityadaptabilitycritical thinkingattention to detail
Certifications
PhD in Learning SciencesPhD in Educational PsychologyPhD in AIEDPhD in MeasurementPhD in Cognitive Science
