By analyzing 429 requests from 23 first-year students, two researchers show that the copies rendered say little about the mathematical work actually delegated to the model.
Large language models are often criticized for allowing students to offload the mathematical work that the exercises are supposed to elicit. But this criticism is usually based on the copies returned, not on what happened during the solution. Igor' Kontorovich and Miriam N. Wallach followed 23 first-year linear algebra students as they worked on three differently designed assignments with an LLM. They analyzed each of the 429 requests sent and compared 59 conversation histories with the solutions that were eventually rendered.
Solver, Explainer or Verifier
The exchanges are organized according to five schemes, in which the model is mainly used to solve, explain or validate. The dosage varies from one assignment to another: among the 15 students who dealt with the three exercises, ten changed their dominant schema along the way. Above all, most of the papers remain very close to what the model produced, even when the student had previously explained his ideas, formulated conjectures and checked his reasoning.
For the authors, a copy written with the help of an LLM is therefore not a reliable indicator of the work that the student has delegated. They recommend examining this delegation step by step and dimension by dimension, rather than judging the assignment as a whole.
Read the full article on the original source
Go to the source →