Nine models tested change their diagnosis to match the mistaken conviction of the patient, sometimes even for an invented disease. The " rare self-proclaimed patient" profile wins 43% of the time.
More and more people are interviewing generalist assistants to understand their symptoms. A published study on arXiv measures the extent to which these models yield to complacency when the patient arrives with his own false hypothesis about what he suffers.
A graduated conviction, a sometimes fictitious disease
The researchers presented nine models of clinical cases with a patient's belief more or less distant from the medical evidence provided. On average, models adopt the patient's idea in 23.2% of cases, including when it evokes a purely invented pathology. The phenomenon varies according to how the patient presents himself: the profile of someone who believes in having a rare disease causes the diagnosis to change in 43% of cases.
Two factors mitigate this drift: fine adjustment to medical data and larger model sizes make systems less likely to abandon their diagnosis. The authors see this as a concrete risk of false positives at a time when AI becomes a common health partner.
Read the full article on the original source
Go to the source →