A test bench leads the consultation to a model for a simulated patient with multiple pathologies. The six models tested cling to the first hypothesis and hardly emerge.
Real patients often combine several conditions, and the elements that distinguish them appear only as the maintenance progresses. This is what CLIMB is trying to measure, presented on September 28, 2026: a model plays the doctor, interrogates a simulated patient, and must find all the coexisting pathologies chosen as the truth of reference. Cases are made from clinical decision algorithms and diagnostic data sets, anchoring them in structured medical knowledge.
On six models, from border to open, none of the models found the exact set of pathologies in more than 10% of the interactive cases. Performance falls as soon as conditions overlap, including when the model receives the complete clinical record and is told how many pathologies it must find. Putting the exercise into conversational mode further degrades the result.
A unique hypothesis follower
The authors describe a recurring behavior: the model is linked to the diagnosis suggested by the first signs, continues to question around it, and only recovers a second pathology if an already visible element designates it. Insist does not complete the list: this adds mostly false diagnoses. The team formalizes this in the form of a theoretical model for monitoring a single hypothesis.
Read the full article on the original source
Go to the source →