Research by Nvidia shows that the same model goes from one to three times depending on the scaffolding that surrounds it. The decisive factor: a supervising agent who reframes the trajectories.
The ARC-AGI-3 benchmark aligns interactive 2D games delivered without notice: the system must discover the rules alone and then win. A research published by Nvidia brings Claude Opus 5 from 30% success, without any particular scaffolding, to 100% once placed in the house harness called AVO. The model has not changed.
Harness refers to all that surrounds the model: the tools it is given, the management of its memory, the rules that frame its actions. Nvidia identifies as a decisive component a supervisor, responsible for detecting trajectory drifts and re-arranging the lead agent before it sinks. On the OpenAI side, the adjustment of two parameters of the harness was sufficient to triple the scores obtained.
The conclusion shifts attention where the industry looks little: with equal gross capacities, the room for manoeuvre available in open scaffolding is wider than is generally assumed. Model rankings, measured without specifying the software envelope used, become all the more difficult to interpret.
Read the full article on the original source
Go to the source →