NeuroVFM: a foundation model trained on 5.24 million routine brain scans

Published in *Nature Medicine*, the model learns directly from MRI and CT scans generated over twenty years of routine clinical practice, and outperforms state-of-the-art models in report writing.

Most medical foundation models are trained on carefully selected public data sets. NeuroVFM takes the lead: it learns directly from 5.24 million series of MRI images and scanners produced by more than 20 years of routine care in Michigan Medicine, without any previous cure. This paradigm is referred to as "health system-wide learning."

Technically, the model is based on Vol-JEPA, a predictive architecture with joint dives adapted to three-dimensional volumes, driven in self-supervision and only on the image — without associated text. It draws a shared representation of the brain anatomy and its pathologies, exploited by diagnostic heads covering 74 diagnoses defined by MRI experts and 82 at the scanner.

Coupled with open language models via light adjustment by visual instruction, NeuroVFM generates radiological reporting that is ahead of advanced models on accuracy, emergency sorting and expert preference. The work is published in Nature Medicine and the code is publicly distributed.