Degraded medical images: MedGemma continues to cut while the visual clues have disappeared, a "mirage" detectable in its activations

By gradually degrading clinical images, researchers identify the point at which a reliable model ceases to be so, and find that it often engages anyway.

Vision-language models for medical imaging are almost always evaluated on clean images, whereas under clinical conditions the available information can degrade. A new study proposes to follow cases that the model solves correctly at the outset, and then deteriorate the image step by step to an empirical "evidence frontier." The authors call the fact that the model continues to give a firm answer at this stage, whether it is right or wrong, "mirage."

From 16% to 100% mirage

Out of three models and three datasets, only MedGemma succeeds in enough cases at the outset to allow analysis. Its behavior across the border is highly variable: pure mirage, recovery of the correct answer, constant abstention or oscillations, with mirage rates ranging from 16% to 100% depending on the dataset/modality pair. On VQA-RAD, the fact that a question can be solved by text alone is related to this persistent commitment (AUROC from 0.653 to 0.670).

The entropy of responses and abstentions do not accurately reflect this phenomenon. On the other hand, probes applied to the internal representations of the model predict whether it will engage or abstain with an AUROC of 0.895 to 0.994. The authors argue for evaluations that take this boundary into account and better abstention mechanisms.