Finding the idea for an article from its bibliography alone: state-of-the-art models succeed in 3 to 15% of cases

The Reconstruction benchmark submits seven models to 643 papers from six disciplines, giving them only the list of references prior to publication.

Posted on arXiv on August 17, 2026, the Benchmark Reconstruction raises an unusual question: can a model reconstruct the idea of researching a scientific article by only having the bibliography that this article cited before it was published? No full text, no author name, no post-publication signal is provided. An independent model then compares the assumptions proposed with the real idea, kept away from the evaluation.

The protocol was designed to withstand the saturation that affects conventional evaluation games: since the references are prior to publication and stripped of any identifying element, a model cannot succeed by finding the answer in its training memory. Seven state-of-the-art models were confronted with 643 articles across six scientific fields.

3-42% by architecture

The correlation rates of individual models range from 3 to 15%. A multi-agent chain, which competes in a Swiss tournament with proposals, criticisms and iterative filtering, climbs to 23-42%, a gain of about 2.4 times on the best single base line – without regaining the majority of ideas. The authors draw a precise reading: a bibliography delimits a space of problems, but the creative leap that links existing literature to a new hypothesis remains exactly what the article brings.