Immediate forecast of rains: too "wet" data sets distort the training and evaluation of generic models

Selecting mostly rainy episodes to train and test models can reverse the diagnosis: the same model seems too cautious or too alarmist depending on the test set.

To predict rain in the next few hours, generative models are generally trained and evaluated on datasets that overrepresent rainy episodes, in order to highlight extreme events, which are rare by nature. A team of researchers shows on arXiv that this choice poses a deeper problem than previously thought.

A bias that hides itself

It was already known that choosing test cases based on observations favors predictions that stick to this selection rather than to the reality of the field. The authors demonstrate mathematically that this sorting also modifies what the model learns. When training and testing are sorted in the same way, even a perfect model learns exactly the distribution that the evaluation rewards, so that the bias of the test masks the bias of the training.

With a diffusion model that converts satellite images into radar data, they show that the verdict can be reversed: the model seems to underestimate the rain on a "wet" test set, but overestimate it on a set that is representative of real conditions of use. The observation is also true seven years after the training period. Taking this verdict at face value can lead developers to the wrong settings, as illustrated by the example of guidance without a classifier.