TutorCaseBench: In the face of students with behavioural difficulties, the best AI tutor only meets one criterion in 55.8% of cases

A Korean benchmark based on case studies in special education shows that LLM know how to recognize the right strategy but struggle to formulate a response adapted to the student.

Language models are increasingly used as tutors, but existing assessments ignore the challenging behaviours common among students with learning or behavioural difficulties. A Korean team, including Juho Kim, built TutorCaseBench from published case studies in pedagogy, educational psychology and special education: each documented student profile is transformed into a scenario, a tutor-student dialogue and an evaluation grid based on the sources.

The game has 923 dialogues across five difficulty types, with 1,560 open-ended items and 2,537 multiple-choice questions. Out of eleven state-of-the-art LLMs, even the best meets the expected criterion in only 55.8% of open-ended cases. Models fare better at choosing the right strategy from proposals than writing a situation-adjusted answer themselves, an even larger gap for open-weight models. The data is published in open access.