FLORE: a humanoid learns to measure its strength in both hands, and achieves 80% of tasks against 27% without stress sensors

Driven on human demonstrations made without robot, with portable clamps that measure the effort, politics pilot both the gestures and the strength of each hand.

Many everyday gestures, such as pushing a piece of furniture or wiping a surface, require controlling the force exerted as well as the movement. On a humanoid, this force must be distributed between the two hands and the whole body, which a simple trajectory does not describe. FLORE, presented on arXiv, tackles this problem of "force-reactive" loco-manipulation.

Robot-free demonstrations

The demonstrations are recorded without a robot: an operator wields two handheld grippers equipped with force sensors, inspired by the UMI device, and wears trackers that follow his body. A diffusion policy receives the image, the state of the robot and the measured forces, then predicts the actions and a force set for each hand. A hybrid whole-body controller executes these instructions while maintaining balance; it corrects the force deviation measured by real force-torque sensors, which allows each hand to follow the requested force in one direction while remaining flexible in the others.

Out of three high-contact tasks conducted on the real robot, the policy passed 80% of the tests, compared to 26.7% for a policy trained on the same demonstrations but controlled only in position, with the same controller.