Adaptive Tutors: Analyzing the way students reason for their failures better than counting their mistakes

Starting with 36,234 student actions in a grammar game, a model identifies four states of reasoning and anticipates success with a AUC of 0.90.

Most adaptive tutoring systems see only one thing: the answer is right or wrong. A study published on arXiv proposes to make the student's progress visible, by analyzing the sequence of his actions in an educational grammar game. The model calculates five readable dimensions of strategy (methodical correction of errors, random answers, etc.), estimates the coordination difficulty imposed by the exercise and classifies reasoning in four distinct states.

Context matters more than profile

Out of 36,234 interactions of 234 students, involving 42 morphosyntax exercises, the strategies depend mainly on the exercise, not much on the student himself (intraclass correlation of 0.14). Students also get bogged down easily: 74.6% of failures related to a high requirement for coordination, rather than to a lack of knowledge, are followed by a new failure if no one intervenes.

The early warning model predicts success with a CUA of 0.826 based on strategies alone, and 0.902 based on the sequential context, without relying on the number of errors in progress. The authors see this as a way to help the student at the right time, according to his or her way of reasoning and not only according to his or her results, and consider the method to be transposable to other disciplines.