Learning Interactive World Model for Object-based Reinforcement Learning
Fan Feng, Phillip Lippe, Sara Magliacane
Agents that can understand and reason about objects and their interactions offer significant potential for robust and generalizable policy learning. While most object-centric reinforcement learning methods focus on object-centric state representations that are factored based on individual objects, they neglect a structured representation of interactions. In this work, we propose Factored Interactive Object-Centric World Model (FIOC-WM), a unified framework for learning structured representations of objects and their interactions within a world model, which can model the dynamics of the environment in a more fine-grained way, enabling more efficient and generalizable policy learning. FIOC-WM first learns object-centric representations with state factorization and object interactions directly from pixels, leveraging pre-trained vision models. Then, we use the learned world model to decompose complex tasks into composable interactions, and learn a hierarchical policy where the high-level decides the interaction order and type, and the low-level learns how to invoke them. We demonstrate the efficacy of FIOC-WM in world model accuracy and policy learning across a set of robotic and embodied AI environments.