Failure Prediction at Runtime without Failure Data for Generative Robot Policies
Ralf Römer, Adrian Kobras, Luca Worbis, Angela Schoellig
Recent advances in imitation learning (IL) with generative models, such as diffusion and flow matching, have significantly improved the capabilities of robots to perform complex, long-horizon tasks. However, distribution shifts from unseen environments or compounding action errors can still cause unpredictable and unsafe behavior, leading to task failure. Therefore, being able to predict failures of generative IL policies as early as possible during runtime is crucial for deploying robots in human-centered and safety-critical environments. The two main existing approaches for recognizing failures both have major drawbacks. Methods relying solely on out-of-distribution (OOD) detection are prone to raising false alarms as policies might generalize. Conversely, external monitoring of the robot’s interactions with its environment provides no foresight about future behavior, meaning that failures can only be detected late or in retrospect. To close the existing gap, we propose FIPER, a framework for Failure Prediction at Runtime for generative IL policies without relying on failure examples. FIPER identifies two key indicators of policy failures: 1) consecutive OOD observations and 2) persistently high uncertainty (entropy) in generated actions. We calibrate both observation- and action-based failure prediction scores on a few successful rollouts and use conformal prediction to provide statistical performance guarantees. We evaluate our framework in five simulation and real-world environments where various types of failures can occur. Our results show that FIPER better distinguishes true failures from OOD situations and predicts failures earlier and more accurately than existing methods. We thus consider FIPER an important step towards more interpretable and safer generative robot policies. Our code and data are available at this link.