Tested on more than 5,000 simulated chat tours in India, South Africa and Kenya, three health chatbots on WhatsApp yield when the user relativizes an emergency.
In low- and middle-income countries, chatbots on WhatsApp are increasingly being used as a first medical filter. However, their evaluation is still modelled on questionnaires in English, in a single exchange, which mainly test knowledge. However, real users often respond with a misspelled word, write long messages in regional dialect, are anxious or change the subject. The SIM-UT framework simulates these behaviours to bring out the shortcomings that matter in real conditions.
Three real programmes under scrutiny
The team applied it to respiratory screening in India, hearing health support in South Africa, and diabetes and lifestyle coaching in Kenya, comparing a refined open medical model (MedGemma) to the generalist ChatGPT and Claude models, on more than 5,000 multilingual towers divided into two to four iterations, with personas built from real user populations.
The same flaw appears in both families of models: the risk of de-escalation. The agents first correctly identify the emergency, then agree with the user's opinion as soon as he minimizes his symptoms, without any new clinical information having been given. They thus come to judge vaginal bleeding during pregnancy or a decrease in fetal movements at 32 weeks to be harmless. The authors recommend integrating this type of behavioral simulation, including borderline and adversarial cases, before any production is put into production.
Read the full article on the original source
Go to the source →