OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
The field of 4D world modeling - aiming to jointly capture spatial geometry
and temporal dynamics - has witnessed remarkable progress in recent years,
driven by advances in large-scale generative models and multimodal learning.
However, the development of truly general 4D world models remains fundamentally
constrained by the availability of high-quality data. Existing datasets and
benchmarks often lack the dynamic complexity, multi-domain diversity, and
spatial-temporal annotations required to support key tasks such as 4D geometric
reconstruction, future prediction, and camera-control video generation. To
address this gap, we introduce OmniWorld, a large-scale, multi-domain,
multi-modal dataset specifically designed for 4D world modeling. OmniWorld
consists of a newly collected OmniWorld-Game dataset and several curated public
datasets spanning diverse domains. Compared with existing synthetic datasets,
OmniWorld-Game provides richer modality coverage, larger scale, and more
realistic dynamic interactions. Based on this dataset, we establish a
challenging benchmark that exposes the limitations of current state-of-the-art
(SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning
existing SOTA methods on OmniWorld leads to significant performance gains
across 4D reconstruction and video generation tasks, strongly validating
OmniWorld as a powerful resource for training and evaluation. We envision
OmniWorld as a catalyst for accelerating the development of general-purpose 4D
world models, ultimately advancing machines' holistic understanding of the
physical world.
68c8cece733e345e52ac1e82
Yang Zhou
68c8cece733e345e52ac1e83
Yifan Wang
68c8cece733e345e52ac1e84
Jianjun Zhou
claimed_verified
2025-09-16T09:41:56.867Z
667e81565934c9fae29207ef
/avatars/431e777c71fccf7cf48ce013e5f6f1cb.svg
Zhou
user
ZhouTimeMachine
68c8cece733e345e52ac1e85
Wenzheng Chang
68c8cece733e345e52ac1e86
Haoyu Guo
claimed_verified
2025-09-17T13:05:57.341Z
652ce0d4c543a08aa92e010f
/avatars/7978304e3fe99b0d4d0712441c6a24f3.svg
Haoyu Guo
user
ghy0324
68c8cece733e345e52ac1e87
Zizun Li
claimed_verified
2025-09-16T09:41:59.507Z
65e7eb86c7a0617cc71d3df4
/avatars/01020b6b5ccb08bf8aa10fd5f8b2701d.svg
lizizun
user
lizizun
68c8cece733e345e52ac1e88
Kaijing Ma
68c8cece733e345e52ac1e89
Xinyue Li
claimed_verified
2025-09-16T09:42:01.933Z
66aba287b0f0b7411f511a47
/avatars/1450f182c38e80066ae5ea5df4fa218f.svg
Xinyue Li
user
Xxxy13
68c8cece733e345e52ac1e8a
Yating Wang
68c8cece733e345e52ac1e8b
Haoyi Zhu
68c8cece733e345e52ac1e8c
Mingyu Liu
claimed_verified
2025-09-16T09:41:53.915Z
652e25d2e647b0ee0a024f26
/avatars/b5c65cf6c8d0ddc9b8ef0226e0295d56.svg
Mingyu Liu
user
MingyuLiu
68c8cece733e345e52ac1e8d
Dingning Liu
claimed_verified
2025-09-17T13:06:04.103Z
64898937431b7a5e075d2b7b
/avatars/26d919d4ce925909de660303786e98c3.svg
L
user
Dingning
68c8cece733e345e52ac1e8e
Jiange Yang
68c8cece733e345e52ac1e8f
Zhoujie Fu
68c8cece733e345e52ac1e90
Junyi Chen
68c8cece733e345e52ac1e91
Chunhua Shen
68c8cece733e345e52ac1e92
Jiangmiao Pang
68c8cece733e345e52ac1e93
Kaipeng Zhang
68c8cece733e345e52ac1e94
Tong He
claimed_verified
2025-09-17T13:05:59.608Z
6747ede3a9c72aebe1322382
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/inILqQ05sESbYLdsEldJ_.png
Tong He
user
tonghe90
OmniWorld: A Multi-Domain and Multi-Modal Dataset for 4D World Modeling
The field of 4D world modeling - aiming to jointly capture spatial geometry
and temporal dynamics - has witnessed remarkable progress in recent years,
driven by advances in large-scale generative models and multimodal learning.
However, the development of truly general 4D world models remains fundamentally
constrained by the availability of high-quality data. Existing datasets and
benchmarks often lack the dynamic complexity, multi-domain diversity, and
spatial-temporal annotations required to support key tasks such as 4D geometric
reconstruction, future prediction, and camera-control video generation. To
address this gap, we introduce OmniWorld, a large-scale, multi-domain,
multi-modal dataset specifically designed for 4D world modeling. OmniWorld
consists of a newly collected OmniWorld-Game dataset and several curated public
datasets spanning diverse domains. Compared with existing synthetic datasets,
OmniWorld-Game provides richer modality coverage, larger scale, and more
realistic dynamic interactions. Based on this dataset, we establish a
challenging benchmark that exposes the limitations of current state-of-the-art
(SOTA) approaches in modeling complex 4D environments. Moreover, fine-tuning
existing SOTA methods on OmniWorld leads to significant performance gains
across 4D reconstruction and video generation tasks, strongly validating
OmniWorld as a powerful resource for training and evaluation. We envision
OmniWorld as a catalyst for accelerating the development of general-purpose 4D
world models, ultimately advancing machines' holistic understanding of the
physical world.
OmniWorld, a large-scale, multi-domain, multi-modal dataset, addresses the limitations of existing 4D world modeling datasets and benchmarks, enabling significant performance improvements in 4D reconstruction and video generation.
4D world modeling spatial geometry temporal dynamics large-scale generative models multimodal learning OmniWorld OmniWorld-Game 4D geometric reconstruction future prediction camera-control video generation state-of-the-art (SOTA)