This agent-based framework extracts spatial memory and the way people approach objects from passive videos. On a real mobile manipulator, it succeeds in 80 per cent of trials where other methods fail.
Videos of daily activities, filmed without the intention of demonstrating, constitute a considerable source of human experience for robots. However, useful knowledge must be extracted from them and adapted to a different body. MoCaP ("mobile code-as-policy"), presented on arXiv, transforms these passive recordings into a guide for a mobile manipulative robot: it draws a memory of the location of objects and clues about how people position themselves before grasping them.
The system combines this information with what the robot observes live and generates, in a closed loop, code that links planning and primitive tools within the geometric and kinematic limits of the machine. On a real robot, indoors, MoCaP passes 80% of the tests on each of the two tasks tested, whereas existing approaches do not. The spatial memory from the videos increases the success of navigation to the object from 70% to 100%, and the imitation of the human approach makes it possible to find an input position that can be reached in all cases from an unfavorable starting point, compared to none without it.
Read the full article on the original source
Go to the source →