Flood mapping: the small Prithvi-Tiny foundation model doesn't beat a U-Net, but runs two and a half times faster on a laptop

Tested on 11 Sen1Floods11 floods, the NASA and IBM model remains slightly behind a conventional architecture in terms of precision, but divides the calculation time by more than half.

Earth observation foundation models promise to map surface water with less annotation effort. Two researchers put this promise to the test with Prithvi-EO-2.0-Tiny, the smallest version of the NASA and IBM model, on the 446 hand-annotated Sentinel-2 tiles of 11 floods in the Sen1Floods11 dataset. They compared it to a comparably sized U-Net, a larger version of Prithvi, and simple spectral index thresholds.

Equivalent accuracy, lower cost

With 252 training tiles, the U-Net achieves an IoU score of 0.856 on water, compared to 0.833 for Prithvi-Tiny refined. Pretraining helps the small transformer well compared to random initialization (+0.046 with 25 thumbnails, +0.023 with 252), without reversing the ranking. Excluding each event in turn, the two approaches are equal (0.768 vs. 0.777, a non-significant difference). Prithvi-Tiny's advantage lies elsewhere: 16.1 ms per 512 × 512 pixel thumbnail on GPU and 258 ms on a laptop processor, compared to 30.6 and 665 ms for the U-Net.

The errors are concentrated on the banks, and the uncertainty of the model indicates a good part of the false pixels, without identifying all of them. The authors conclude that these compact models are justified by their sobriety of calculation in emergency situations, not by systematic superiority.