Diabetic retinopathy: Retinal camera brand varies the performance of a foundation model

Out of 8,524 Brazilian patients, images of a Nikon retinograph were twice as often inoperable as those of a Canon. A study shows that this gap has an uneven impact on 12 AI diagnostic tasks.

Diabetic retinopathy is one of the leading causes of preventable blindness, and AI is often presented as a way to extend still difficult screening in many countries. But its performance depends on the quality of the images. Lisa Y. W. Tang looked at a Brazilian cohort of 8,524 patients: the shots taken with a Nikon NF5050 retinograph were more than twice as likely to be of insufficient quality as those of a Canon CR-2, even though a reconditioned Canon costs more than double a Nikon. A cost-quality arbitration that directly affects fair access to screening.

The study then assesses, using a pre-entered and frozen retinal foundation model, whether these device-related deviations are reflected in the results on 12 disease classification tasks, with four input resolutions and five different graders. Differences vary greatly from task to task, and neither higher resolution nor the choice of classifier systematically reduce them. For the author, the device type is a source of too often neglected variability, to be taken into account before deploying these tools to heterogeneous equipment parks. The code and detailed results are published on GitHub.