RAPTOR: A Foundation Policy for Quadrotor Control
Humans are remarkably data-efficient when adapting to new unseen conditions,
like driving a new car. In contrast, modern robotic control systems, like
neural network policies trained using Reinforcement Learning (RL), are highly
specialized for single environments. Because of this overfitting, they are
known to break down even under small differences like the Simulation-to-Reality
(Sim2Real) gap and require system identification and retraining for even
minimal changes to the system. In this work, we present RAPTOR, a method for
training a highly adaptive foundation policy for quadrotor control. Our method
enables training a single, end-to-end neural-network policy to control a wide
variety of quadrotors. We test 10 different real quadrotors from 32 g to 2.4 kg
that also differ in motor type (brushed vs. brushless), frame type (soft vs.
rigid), propeller type (2/3/4-blade), and flight controller
(PX4/Betaflight/Crazyflie/M5StampFly). We find that a tiny, three-layer policy
with only 2084 parameters is sufficient for zero-shot adaptation to a wide
variety of platforms. The adaptation through In-Context Learning is made
possible by using a recurrence in the hidden layer. The policy is trained
through a novel Meta-Imitation Learning algorithm, where we sample 1000
quadrotors and train a teacher policy for each of them using Reinforcement
Learning. Subsequently, the 1000 teachers are distilled into a single, adaptive
student policy. We find that within milliseconds, the resulting foundation
policy adapts zero-shot to unseen quadrotors. We extensively test the
capabilities of the foundation policy under numerous conditions (trajectory
tracking, indoor/outdoor, wind disturbance, poking, different propellers).
68caf6d95a7803ff3be42c7c
Jonas Eschmann
68caf6d95a7803ff3be42c7d
Dario Albani
68caf6d95a7803ff3be42c7e
Giuseppe Loianno
RAPTOR: A Foundation Policy for Quadrotor Control
Humans are remarkably data-efficient when adapting to new unseen conditions,
like driving a new car. In contrast, modern robotic control systems, like
neural network policies trained using Reinforcement Learning (RL), are highly
specialized for single environments. Because of this overfitting, they are
known to break down even under small differences like the Simulation-to-Reality
(Sim2Real) gap and require system identification and retraining for even
minimal changes to the system. In this work, we present RAPTOR, a method for
training a highly adaptive foundation policy for quadrotor control. Our method
enables training a single, end-to-end neural-network policy to control a wide
variety of quadrotors. We test 10 different real quadrotors from 32 g to 2.4 kg
that also differ in motor type (brushed vs. brushless), frame type (soft vs.
rigid), propeller type (2/3/4-blade), and flight controller
(PX4/Betaflight/Crazyflie/M5StampFly). We find that a tiny, three-layer policy
with only 2084 parameters is sufficient for zero-shot adaptation to a wide
variety of platforms. The adaptation through In-Context Learning is made
possible by using a recurrence in the hidden layer. The policy is trained
through a novel Meta-Imitation Learning algorithm, where we sample 1000
quadrotors and train a teacher policy for each of them using Reinforcement
Learning. Subsequently, the 1000 teachers are distilled into a single, adaptive
student policy. We find that within milliseconds, the resulting foundation
policy adapts zero-shot to unseen quadrotors. We extensively test the
capabilities of the foundation policy under numerous conditions (trajectory
tracking, indoor/outdoor, wind disturbance, poking, different propellers).
A method called RAPTOR enables a single neural network policy to adapt zero-shot to various quadrotors using Meta-Imitation Learning and In-Context Learning with recurrence in the hidden layer.
Reinforcement Learning RL Simulation-to-Reality Sim2Real RAPTOR In-Context Learning Meta-Imitation Learning trajectory tracking wind disturbance