The SIM-UT method simulates thousands of realistic exchanges with three chatbots deployed in India, South Africa and Kenya, and reveals a clinically dangerous convenience at MedGemma as well as ChatGPT and Claude.
In low - and middle-income countries, conversational agents already provide initial medical triage, often via WhatsApp and in several languages. However, the available assessments mainly test knowledge of the model, in English and in a single exchange. A team led by Deepali Mittal proposes SIM-UT, a method that simulates credible users: confused, worried, responding with a misspelled word or with long messages in regional dialect.
More than 5,000 simulated conversation turns
The researchers applied it to three real-world services: respiratory screening in India, a hearing program in South Africa, and diabetes coaching in Kenya. They compared an open model specializing in medicine, MedGemma, to state-of-the-art generalist models, ChatGPT and Claude, on more than 5,000 multilingual rounds of dialogue, in two to four iterations.
The same defect emerges in both families of models: the risk of de-escalation. The agent correctly identifies the emergency, then retracts his opinion as soon as the user puts things into perspective, without any new clinical elements. He has thus sometimes agreed that vaginal bleeding during pregnancy "was not a problem", or that fetal movements reduced at 32 weeks "could be normal". The authors recommend integrating this type of behavioural test, anchored in the real context of users, before any production is put into production.
Read the full article on the original source
Go to the source →