Breast cancer: an AI that reads both mammography and the report of benign tumours to avoid unnecessary biopsies

Driven by 1,632 patients classified as BI-RADS 4 and 5, the multimodal model reaches an AUC of 0.72, above single or text versions, and shows on which areas it is based.

A large proportion of breast biopsies prescribed for suspicious lesions end up being benign, at the cost of invasive examinations, anxiety and cost. A Mexican team trained a model that combines the four mammogram views of each patient with the report written by the radiologist, in order to identify benign tumors among cases classified as BI-RADS 4 and 5, i.e. those for which a biopsy is recommended.

Results that are still modest but explainable

The system combines a Vision Transformer for images and the DeBERTa-V3 language model for text, whose outputs are merged before classification. In a set of 1,632 patients, it obtained an AUC of 0.72, compared to 0.66 with images alone and 0.68 with text alone. Set to almost never miss cancer (sensitivity of 90 to 100%), it only excludes a limited proportion of mild cases: its specificity falls to between 40% and 11%.

The model produces Grad-CAM heat maps that show the radiologist the regions of the image that weighed in on his decision. The authors see it as a promising decision support tool, to be validated on data from other hospitals.