Breast ultrasound: multimodal models tested on 2,818 images still underperform a naive classifier on a dataset

A study compares CLEF and CLEF-Flash using three public breast ultrasound datasets: accuracy varies significantly depending on the prompt and the dataset, with numerous false-positive diagnoses of malignancy.

Can the classification of mammary ultrasounds be entrusted to multimodal generalist models without retraining? A study published on arXiv evaluated two of them, CLEF and CLEF-Flash, on 2,818 images from three public databases (Dataset B, BUSI and BUS-BRA), with three set-up formulations: simple list of labels, explicit classification task, or clinical ultrasound criteria.

Results highly dependent on context

CLEF-Flash associated with the clinical setting obtains the best accuracy on all three bases: 74.85 % on Dataset B, 66.28 % on BUSI and only 51.63 % on BUS-BRA. On BUSI, however, CLEF with task-oriented instructions gets the best balanced scores. Clinical guidelines improve CLEF-Flash but degrade CLEF on all bases; CLEF-Flash is always the fastest.

Confusion matrices reveal frequent malignantly classified benign lesions. On BUS-BRA, no configuration exceeds 67.63% that would be obtained by a predictor systematically responding to the majority class. The authors recommend class-by-class assessment before using these models in breast ultrasound.