In the absence of numerous robotic demonstrations, Being-M0.7 first learns the dynamics of motion from more than 10,000 hours of human data, then translates it to the control of a humanoid's entire body.
Walking while manipulating objects requires a humanoid to anticipate the evolution of the scene and coordinate its entire body. Robot demonstrations are rare. Videos and human motion capture abound, but they often provide the image without the gesture or the other way around, and human movement does not translate directly into commands for a robot.
Three steps, from human to robot
Being-M0.7 is a latent world-action model trained in three stages. The pre-training is based on a corpus of more than 10,000 hours of human-centric data, mixing videos alone, movements alone and video-motion pairs; by predicting future visual states and movement together, the model learns representations that encode future kinematics. An intermediate phase adapts it to the robot's point of view and morphology. Finally, an "action expert" module combines these predictive, frozen representations with the current image and proprioception to produce executable whole-body commands.
The model has the best overall success rate among the approaches compared on the SIMPLE simulated test bench, and equals the best of them on real loco-manipulation tasks conducted with a Unitree G1 humanoid.
Read the full article on the original source
Go to the source →