ColoACT: a self-propelled endoscopic robot is learning to navigate the colon on its own

Tested on ex vivo pig colons, the system successfully completes 85.4 per cent of straight-line journeys and 70 per cent of 90-degree turns thanks to a policy that predicts sequences of actions.

Autonomous colonoscopy may reduce operator load and the risk of loops or injury, but the colon is a hostile environment for computer vision: deformable walls, little texture, specular reflections, and constant contact. A team of seven researchers presents ColoACT, a navigation system for a small, self-propelled endoscopic robot with bevel gears.

Depth, pseudo-relief and block actions

The steering policy is based on an "action chunking" transformer: rather than deciding one movement at a time, it predicts overlapping sequences of actions and then merges them over time, making the movement smoother. As input, the color image is enriched with a relative depth estimate and a pseudo-elevation map calculated from the gradients, which highlights the folds of the wall.

On segments of about 60 cm of pig colon ex vivo, the robot achieves 85.4% success in a straight line and 72.5% in curved sections, 70% on 90-degree turns and 60% on double curves. The authors also show the feasibility on triple curves, which are more demanding.