Thinking, careful, rash or collaborative: a validated scale of 2,700 responses to measure how students rely on AI

Researchers propose a questionnaire that describes the use of generative AI during problem solving, without judging it by a good or bad response.

Measuring AI "dependence" often comes down to checking whether the user has followed a right or wrong suggestion. This logic doesn't work well for complex problem-solving, where there isn't always a single answer or immediate verdict. A team publishes a scale on arXiv that instead describes the behaviors of undergraduates when they use generative AI.

An exploratory factor analysis conducted on 800 responses, collected after a problem-solving activity, revealed four usage profiles: thoughtful, cautious, thoughtless and collaborative. The scale shows good internal consistency (Cronbach's alpha of 0.84) and was confirmed on 730 other responses from the same activity, then on 1,173 responses from a second activity. The authors see it as a tool to help teachers identify uses and adapt their integration of generative AI in higher education.