AMIE tested prior to live consultations: 98 supervised interactions, no safety shutdowns

BIDMC is evaluating a medical chatbot within a real-world care pathway. The results support its feasibility under supervision, although they do not demonstrate any health benefits.

BID™ and Google researchers studied AMIE with patients preparing for an urgent primary care consultation. Out of 114 people recruited, 98 completed the exchange and consultation. A doctor monitored each conversation in real time and could intervene.

None of the exchanges required a safety shutdown, but a hallucination was identified and five cases required clinical clarification. The researchers point out that the study published in The Lancet assesses feasibility, not improvement in health outcomes. Reservations about privacy and trust persist. Alphabet funded the work; this close supervision limits extrapolation to an autonomous chatbot.