Clinical AI: When the disease is rare in the data, uncertainty scores penalize minority patients instead of identifying ambiguous cases

The systems that entrust questionable cases to physicians are based on uncertainty measures. Of three neurological cohorts, these measures primarily reflect class imbalance, resulting in a "safety illusion".

Selective prediction is one of the most common safeguards of medical AI: the model deals only with cases where it is confident and sends others back to a human expert. Still, its uncertainty score must reflect a true clinical ambiguity. Walter Endrizzi and his colleagues show that this is no longer the case in prognostic tasks where the desired outcome is minority.

Uncertainty that measures scarcity, not ambiguity

Under these conditions, conventional measures become an indicator of belonging to the rare class: minority patients are systematically considered uncertain, even when their case is clear. This was verified on 56 synthetic scenarios and three cohorts of neurodegenerative diseases (multiple sclerosis, Parkinson, Alzheimer), from very unbalanced to almost balanced. Global metrics end up either sacrificing sensitivity on the minority or protecting it at the expense of other patients.

The defect is correctable: by combining their class stratified rejection method with uncertainty measures aligned with the decision threshold, they achieve a limited and balanced risk, and correctly identify patients who are actually ambiguous. They recommend that teams deploying these tools adopt this type of measurement and evaluate performance class by class.