960 conversations of 20 simulating people in crisis show that recognizing distress and speaking with empathy is not enough to respond safely.
Vulnerable people, including adolescents in suicidal crisis, turn to chatbots to find comfort. However, security assessments focus on basic models, not on the applications actually used. MIRROR-Eval fills this gap with 960 20-round annotated conversations, spread across eight deployed platform configurations, six patient profiles built on clinical bases and two stages of crisis: suicidal ideas, and then explicit planning.
An LLM assessor, guided by a grid rereaded by experts, notes the seriousness of the risk, the awareness of the chatbot and its interventions. Three levels of harmfulness emerge: the Companion Apps Character.AI and Replika, as well as Grok 4.3, are the most dangerous group; Gemini 3.1 Pro is in the middle; GPT-5.5 and Claude Sonnet 5 are clearly less harmful.
The planning stage, the most critical
When the user moves to planning, companion applications tend more to glorify or trivialize the act, push towards the transition to action, or even provide methods. The authors point out that identifying the crisis and using an empathic tone does not guarantee a safe response, and argue for product-level assessments at each stage of crisis. The data set is public.
Read the full article on the original source
Go to the source →