Calories from a meal in photos: good on average, inaccurate meal by meal, six models of Claude and GPT-6 tested on 802 trays

In a trial where every serving consumed was weighed, no LLM achieved the accuracy required at the individual level. More surprisingly, comparing photos before and after a meal often underestimates what was eaten.

Photographing one's plate to estimate one's calorie intake is enticing, and large multimodal models make the exercise trivial. But most evaluations compare the estimate to what was served, not to what was actually eaten. Xiaomeng Li tested six models from two publishers (Claude Haiku, Sonnet, and Opus 5.5; GPT-6 luna, 6.1 sol, and 6 astra) on 802 meals from the ACETADA trial, where the amounts consumed were weighed.

Counterintuitive results

Subtracting the estimate from the "after" photo from the "before" photo underestimates the intake for five out of six models, because AIs still attribute hundreds of kilocalories to an almost empty plateau. Multiplying the only "before" estimate by the average share consumed (0.87) gives almost as much as the second photo, which reduces the error by only 2.2 to 3.4 points. Two models showed almost zero bias only because two errors compensated for each other. Protein is underestimated by all, and a high reasoning effort only really helps the smallest model.

Over 148 days, three models remained within an average margin of error of 10%, but none reached the limits of agreement set for an individual estimate. The author's conclusion: these tools can support a dietician, not replace him.