Humanoids: simple user requests are enough to foil their anti-collision filters

Safety filters ensure that there is no collision only if properly configured. The SIREN tool finds trapped targets that impact a real humanoid.

To operate in homes, hospitals or warehouses, humanoid robots rely increasingly on "safety filters", which provide a mathematical guarantee that the generated controls will not cause a collision. A team of researchers shows that this guarantee depends entirely on setting parameters specific to each task and environment, for which no general method exists. A filter delivered with a robot can therefore fail once it is deployed in certain situations.

The authors describe attacks by inserting an objective: without particular privilege, only through the normal interactions expected for the user, an attacker selects targets that lead the robot in the blind corners of the filter. Their automated tool, SIREN, identifies such attacks in all vulnerable scenarios of the SPARK dataset, and discovers three times more than a reference method. It also resulted in a collision of a real humanoid while performing his duties.

The conclusion is uncomfortable: mechanisms designed to protect people around robots can themselves become a source of risk.