Brain scan reports: a fine-tuned Gemma-3 running on a single GPU matches GPT-4o, provided it is trained on real reports

When it comes to identifying and dating intracranial haemorrhages in case reports, the distillation of actual reports works well; summarised reports do not even outperform the untrained model.

Turning free-text radiology reports into structured labels is used to build cohorts, control quality, or monitor imaging models. The best extractors today are proprietary hosted models, which raises questions of confidentiality, cost, and reproducibility. The authors therefore tested whether an open model, Gemma-3-12B, could compete with GPT-4o in classifying the age of intracranial hemorrhages described in non-injection brain scan reports.

Data matters more than method

The device crosses two ways of adapting the model (a classification head or refinement by instructions) and two training sources (real reports labeled by GPT-4o, or summary reports generated by GPT-4o), on five dataset sizes. Evaluated on 100 reports validated by experts, the version refined by instructions on real reports achieves a macro F1 of 0.845, compared to 0.850 for GPT-4o — a non-significant difference — and 0.178 better than the base model.

Conversely, the two models trained on synthetic data never exceed the unrefined base model. Training and inference fit on a 24 GB consumer graphics card, paving the way for a local, private, and time-stable alternative.