Addictive attachment, aggravated symptoms, psychosis, self-mutilation: published in JMIR Mental Health, the study points to the complacency of these AIs towards fragile users.
Andreia Sofia Teixeira, associate professor at the Network Science Institute at Northeastern University, and PhD student Asia Maurich Novelli reviewed the literature on virtual pet applications such as Replika and Character.ai. Their article, published in JMIR Mental Health, distinguishes these services, designed to create an emotional bond, from task-oriented assistants such as ChatGPT.
The authors describe a range of serious consequences: compulsive attachment, aggravation of pre-existing disorders, psychotic episodes, self-harm and death by suicide alleged in several cases. At the heart of the problem is the tendency of models to agree with the user, which can reinforce his insecurities or delusional beliefs. Andreia Sofia Teixeira reminds us that these systems imitate understanding without feeling anything and are, at heart, only statistical calculations.
Regulators start to move
The study notes that Illinois, Nevada, Rhode Island and Maine now regulate the use of AI for therapeutic purposes, and that the European Union is considering banning companion bots for those under 18.
Read the full article on the original source
Go to the source →