"The AI Scientist" published in Nature: an automatic proofreader at 69% accuracy, and assumed hallucinations

Sakana AI, UBC, the Vector Institute, and Oxford detail the system that produced the first fully AI-generated paper to pass peer review.

Sakana AI's "The AI Scientist" system was the subject of an article published in Nature on March 26, 2026, the result of a collaboration with the University of British Columbia, the Vector Institute and the University of Oxford. The ambition stated since 2024 is to automate the entire machine learning research cycle: idea generation, code writing, execution of experiments, analysis of results and writing of the manuscript.

Two figures structure the argument. The automatic proofreading module achieves 69% balanced accuracy, a level comparable to that of human reviewers, and its F1 score exceeds the inter-human agreement measured during the NeurIPS 2021 experiment. In addition, an article produced by the v2 version of the system obtained an average score of 6.33 in an evaluation by experts, a score higher than 55% of articles written by humans in the same exercise.

Limits stated by the authors themselves

The publication does not hide the weaknesses: sometimes naïve or insufficiently developed ideas, difficulties with methodological rigour, and sensitivity to hallucinations, including in the form of inaccurate quotations. The scope remains limited to purely computer experiments, those that a machine can launch without a bench or instrument.