From Masks to Worlds: A Hitchhiker's Guide to World Models
This is not a typical survey of world models; it is a guide for those who
want to build worlds. We do not aim to catalog every paper that has ever
mentioned a ``world model". Instead, we follow one clear road: from early
masked models that unified representation learning across modalities, to
unified architectures that share a single paradigm, then to interactive
generative models that close the action-perception loop, and finally to
memory-augmented systems that sustain consistent worlds over time. We bypass
loosely related branches to focus on the core: the generative heart, the
interactive loop, and the memory system. We show that this is the most
promising path towards true world models.
68fb1095f158a71c5a2f58dd
Jinbin Bai
claimed_verified
2025-10-24T16:01:08.585Z
63fccdac93b993a4ebd7789a
https://cdn-avatars.huggingface.co/v1/production/uploads/63fccdac93b993a4ebd7789a/KRx8vpdoDjsZBRw0j8Vg8.jpeg
Jinbin Bai
user
BryanW
68fb1095f158a71c5a2f58de
Yu Lei
68fb1095f158a71c5a2f58df
Hecong Wu
68fb1095f158a71c5a2f58e0
Yuchen Zhu
68fb1095f158a71c5a2f58e1
Shufan Li
68fb1095f158a71c5a2f58e2
Yi Xin
68fb1095f158a71c5a2f58e3
Xiangtai Li
68fb1095f158a71c5a2f58e4
Molei Tao
68fb1095f158a71c5a2f58e5
Aditya Grover
68fb1095f158a71c5a2f58e6
Ming-Hsuan Yang
From Masks to Worlds: A Hitchhiker's Guide to World Models
This is not a typical survey of world models; it is a guide for those who
want to build worlds. We do not aim to catalog every paper that has ever
mentioned a ``world model". Instead, we follow one clear road: from early
masked models that unified representation learning across modalities, to
unified architectures that share a single paradigm, then to interactive
generative models that close the action-perception loop, and finally to
memory-augmented systems that sustain consistent worlds over time. We bypass
loosely related branches to focus on the core: the generative heart, the
interactive loop, and the memory system. We show that this is the most
promising path towards true world models.
The guide outlines a progression from early masked models to memory-augmented systems, emphasizing generative capabilities, interactive loops, and memory for building world models.
masked models representation learning unified architectures interactive generative models action-perception loop memory-augmented systems world models