By generating counterfactual variants of recorded movements and assessing the physical plausibility of the predicted videos, DreamTrue reduces the interaction failure rate from 48 per cent to 6 per cent.
Robotic world models predict in video what will happen if the robot performs a given action. Two flaws lie in wait for them when they are trained on existing datasets: an inaccurate calibration of the cameras that prevents them from faithfully following the requested action, and an overrepresentation of successful attempts that leads them to predict success even when the gesture should fail. DreamTrue, presented on arXiv by a team of nine researchers, tackles both.
The model, which is multi-view and can be used with several types of robots, projects the action trajectories into the image and aligns them with offline geometric calibration. The authors then modify the recorded trajectories to generate "counterfactual" futures, covering more contacts and failures. In the absence of corresponding real videos, a reward model trained on videos annotated by humans (defects of the robot, objects, interactions) guides post-training by reinforcement. On AgiBot, the rate of interaction defects judged by humans drops from 48.12% to 6.25%, and DreamTrue comes first in the world model category of the AgiBot World 2026 challenge.
Read the full article on the original source
Go to the source →