A new benchmark test asks researchers to build a stochastic ENSO model from real observations within six hours; six out of twelve systems outperform the benchmark.
How do you judge AI-led research when no one knows the right answer? A team of seven researchers proposes SciExam for ENSO, an exam focused on the El Niño Southern Oscillation, the main mode of climate variability from one year to the next. Agents have six hours to process the observations, write their own diagnoses, which are then frozen, and build a simplified stochastic model guided solely by these diagnoses.
Hidden correctors then check whether the model reproduces ENSO statistics, finds unobserved variables, and predicts years set aside; a published model is scored according to the same grid. Out of twelve agent systems tested, six score better than this benchmark, mainly due to better reconstruction and forecasting.
Notably, in their simplified form, the best models each lean towards one of two competing explanations for ENSO's asymmetry between hot and cold phases, an open scientific debate that the statement never mentioned. Controlled trials suggest that these performances do not come from memorizing historical data.
Read the full article on the original source
Go to the source →