“What if the automation of AI R&D were to trigger an explosion of intelligence?”: researchers consider the hypothesis serious enough to warrant legislation

The systems already write most of the code in the laboratories where they are developed. The authors believe that the entire research pipeline could be automated within a few years.

The starting point of this study is an observation in the field: in the companies that build the models, it is now the models that write most of the code — a situation that was unparalleled a year earlier. The authors draw a direct question: if the AI research and development chain is automated from end to end, can progress be compressed to the point that years of progress can be made in a few months?

Their answer is cautious but not reassuring. Preliminary evidence, they write, points in this direction; most of the AI R&D work could be automated within a few years, perhaps all of it. The consequences are symmetrical: massive acceleration of profits on the one hand, and on the other a growth of capacities faster than the ability of societies to adapt, a possible loss of control over superhuman systems, and the erosion of institutional checks and balances between states, companies and branches of power.

Three requests to decision-makers

The authors do not conclude that it is inevitable — they insist on the degree of uncertainty — but consider the stakes to be sufficient to act without delay. They call for real visibility on the state of automation of AI R&D, the development of levers to guide and contain such an acceleration, and the preparation of societies for its effects.