This neurosymbolic approach predicts the actions to come from a short video, and corrects its predictions by editing the knowledge base, without re-training.
To collaborate with humans, a robot must guess what happens next from a few seconds of observation. Purely neural models need long sequences to do this and remain difficult to interpret or correct. SymAnt offers a hybrid pathway, which combines neural networks and structured reasoning.
A symbolic context as a point of support
Starting with a short video, the system jointly explores a scene graph and a knowledge graph in order to construct a symbolic context: spatial relationships, object attributes, possible uses and associated actions. This context serves as a starting point for an iterative process that generates and then refines the sequence of future actions.
The authors report good results on five datasets of activity videos (cooking, assembly), especially when the observation window is short, with a reduced number of parameters. Practical advantage: a user can directly modify the underlying knowledge to adjust the predictions, without new training.
Read the full article on the original source
Go to the source →