The LongChang test bench drops language models in simulations where, for example, heavy objects fall slower. GPT-6-Astra identifies anomaly in 16.7% of cases; The other eleven, never.
A scientific discovery often begins with an unexpected observation. Zhen Yang, Ying-Cong Chen and their co-authors wanted to find out if large language models can notice such an anomaly, and then study it, without being told the question. They designed LongChang, a system that generates simulated worlds where one or more laws of physics differ from reality.
Explore without instructions
The model being tested plays the role of explorer: it manipulates objects, makes tools and conducts its own experiments, without tasks or clues. Other agents generate and verify the scenes, examine the designed tools and judge the final reports in the light of modified laws and traces of exploration, which also serve to make the scenes evolve. The cycle runs without human intervention.
Out of twelve models, only GPT-6-Astra discovers a modified law, in 16.7% of cases; all the others remain at zero. Exploration logs show anomalies observed but never investigated, and laws identified only in part: autonomous scientific curiosity remains a weak point of current models.
Read the full article on the original source
Go to the source →