By combining reinforcement learning with the imitation of eight specialised policies, researchers are training a single multi-task controller, deployed on a Unitree B1 quadruped.
Reinforcement learning allows legged robots to master isolated skills, but uses such as agriculture or space exploration require a varied repertoire: moving, digging, inspecting closely. Training a single do-it-all policy comes up against the inefficiency of samples and the conflicts of gradients between tasks.
Three steps and tasks chosen by adversity
The authors first train several "teacher" policies, each on a narrow task. A "student" model then learns from these teachers by combining reinforcement and imitation, with a mechanism that focuses the training on the task where it performs the least. The result: a student from eight teachers performs 22 tasks, including compound behaviors such as crawling, from walking, digging, and jumping.
According to the paper, the method better preserves the quality of movements and tracks commands more accurately than reference approaches (PPO, distillation followed by fine-tuning), and sometimes generalizes to tasks not seen in training. The policy was deployed in real-world conditions on a Unitree B1 quadruped.
Read the full article on the original source
Go to the source →