TIDE: A handling robot learns during deployment, from human corrections, in less than an hour

This system continuously updates a robot policy based on the interventions of an operator and his own tests, without criticism learned or reward model.

Large foundation models for robotics offer a good foundation for manipulation, but adapting them to a specific task with pre-recorded remotely operated demonstrations often leaves out the situations that matter in real-world conditions: delicate contacts, resuming mistakes, precision gestures. Existing methods add data from human interventions or use e-reinforcement learning, at the cost of slow adaptation in the first case and increased complexity (value functions, actor-critical models) in the second.

Learning from who produced each gesture

One team presents TIDE (Training In DEployment), an online post-training system with a human in the loop. It updates the robot's policy during deployment, based on operator interventions and trajectories produced by the robot itself, in a supervised learning framework that takes into account the origin of each piece of data. It dispenses with learned criticism, reward modeling, and action scoring.

Out of four real-world two-arm manipulation tasks, TIDE improves both task progress and complete success rate compared to the original policy and adding equivalent offline demos, while making new versions available as deployment progresses. Each session, from the first try to the latest version of the model, lasts less than an hour, with learning and versioning running in parallel.