624 episodes on 52 domains to assess whether a model can anticipate future research directions — with model rankings that thwart expectations.
On September 1, 2026, Fenghai Li, Zihan Tang, Haofei Yu, Yining Zhao and Jiaxuan You published a work that asks an unusual question: can a language model predict the topics that a scientific community will then address? The protocol is standardized. The system is given the literature of a field up to a cut-off date, it proposes up to five classified ideas, and these proposals are compared with the articles actually published afterwards.
The benchmark, called the IdeaForecastBench, includes 624 iterative episodes across 52 thematic areas. The authors compare five historical compression strategies on GPT-4.1, Qwen2.5-7B/14B, and Qwen3.5-9B, as well as a trained predictor, the Mode-Decomposition Forecaster.
Summarizing the literature rather than providing it raw improves Hit@5 and Precision@5 scores on the four models tested. The ranking, on the other hand, is surprising: Qwen2.5 is ahead of GPT-4.1 where Qwen3.5 remains inferior. The authors immediately temper the interpretation — Qwen2.5 produces broader predictions, and seeing an idea come to fruition does not prove that it was accurately anticipated.
Read the full article on the original source
Go to the source →