DiffDreamer: a racing drone that separates what he imagines from the world and what he knows from his own physics

This method combines a learned world model, charged with anticipating mobile obstacles, with a differentiable quadrirotor simulator, and converges faster than existing approaches in mobile door racing.

Passing through doors that move at full speed forces a drone to predict the movements of its environment while maintaining a very precise, high-frequency piloting. Reinforcement learning based on a world model can imagine the future, but it usually mixes the evolution of the scene and the dynamics of the robot in a single prediction, so that the errors of the model penalize the training of politics.

Two prediction engines in parallel

Fanxing Li and his co-authors propose DiffDreamer, which divides up roles: the learned model takes care of the external elements and the history, while a differentiable simulator explicitly calculates the physical state of the quadcopter. The objective of the policy is also divided, between a critically-guided path for non-derivable rewards (going through the door) and a path through the simulator for derivable movement rewards.

In aggressive target tracking as well as in moving door racing, the method converges faster than modelless, modeled or purely differentiable approaches, and also exceeds conventional planning and control in terms of final performance. Real flights validate the principle, as a proof of concept.