Aether: Geometric-Aware Unified World Modeling
The integration of geometric reconstruction and generative modeling remains a
critical challenge in developing AI systems capable of human-like spatial
reasoning. This paper proposes Aether, a unified framework that enables
geometry-aware reasoning in world models by jointly optimizing three core
capabilities: (1) 4D dynamic reconstruction, (2) action-conditioned video
prediction, and (3) goal-conditioned visual planning. Through task-interleaved
feature learning, Aether achieves synergistic knowledge sharing across
reconstruction, prediction, and planning objectives. Building upon video
generation models, our framework demonstrates unprecedented synthetic-to-real
generalization despite never observing real-world data during training.
Furthermore, our approach achieves zero-shot generalization in both action
following and reconstruction tasks, thanks to its intrinsic geometric modeling.
Remarkably, even without real-world data, its reconstruction performance far
exceeds that of domain-specific models. Additionally, Aether leverages a
geometry-informed action space to seamlessly translate predictions into
actions, enabling effective autonomous trajectory planning. We hope our work
inspires the community to explore new frontiers in physically-reasonable world
modeling and its applications.
67e22eca9455abdd1d257263
Aether Team
67e22eca9455abdd1d257264
Haoyi Zhu
admin_assigned
2025-03-25T09:15:13.586Z
6283546209aa80237c6c482c
/avatars/0d6fc5846c0456d5282d82d5bf4d7056.svg
Haoyi Zhu
user
HaoyiZhu
67e22eca9455abdd1d257265
Yifan Wang
67e22eca9455abdd1d257266
Jianjun Zhou
claimed_verified
2025-03-25T09:05:54.719Z
667e81565934c9fae29207ef
/avatars/431e777c71fccf7cf48ce013e5f6f1cb.svg
Zhou
user
ZhouTimeMachine
67e22eca9455abdd1d257267
Wenzheng Chang
claimed_verified
2025-03-25T08:20:10.947Z
67a5b0fe5a8652514e67c38c
/avatars/28da8e93ee00fd77c7e62d16f9b94045.svg
Wenzheng Chang
user
AmberHeart
67e22eca9455abdd1d257268
Yang Zhou
67e22eca9455abdd1d257269
Zizun Li
admin_assigned
2025-03-25T09:16:06.912Z
65e7eb86c7a0617cc71d3df4
/avatars/01020b6b5ccb08bf8aa10fd5f8b2701d.svg
lizizun
user
lizizun
67e22eca9455abdd1d25726a
Junyi Chen
admin_assigned
2025-03-25T09:16:13.751Z
6679bb85972a0f224cde335c
/avatars/bf0649645458e206ba5224b001723641.svg
Junyi Chen
user
Junyichen
67e22eca9455abdd1d25726b
Chunhua Shen
67e22eca9455abdd1d25726c
Jiangmiao Pang
admin_assigned
2025-03-25T09:16:26.855Z
65783ee6ee33d547aecc3ffc
https://cdn-avatars.huggingface.co/v1/production/uploads/65783ee6ee33d547aecc3ffc/lWZX88c-0dCsN-yB9Jhlf.jpeg
Jiangmiao Pang
user
Jiangmiao
67e22eca9455abdd1d25726d
Tong He
claimed_verified
2025-07-21T08:33:15.026Z
6747ede3a9c72aebe1322382
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/inILqQ05sESbYLdsEldJ_.png
Tong He
user
tonghe90
Aether: Geometric-Aware Unified World Modeling
The integration of geometric reconstruction and generative modeling remains a
critical challenge in developing AI systems capable of human-like spatial
reasoning. This paper proposes Aether, a unified framework that enables
geometry-aware reasoning in world models by jointly optimizing three core
capabilities: (1) 4D dynamic reconstruction, (2) action-conditioned video
prediction, and (3) goal-conditioned visual planning. Through task-interleaved
feature learning, Aether achieves synergistic knowledge sharing across
reconstruction, prediction, and planning objectives. Building upon video
generation models, our framework demonstrates unprecedented synthetic-to-real
generalization despite never observing real-world data during training.
Furthermore, our approach achieves zero-shot generalization in both action
following and reconstruction tasks, thanks to its intrinsic geometric modeling.
Remarkably, even without real-world data, its reconstruction performance far
exceeds that of domain-specific models. Additionally, Aether leverages a
geometry-informed action space to seamlessly translate predictions into
actions, enabling effective autonomous trajectory planning. We hope our work
inspires the community to explore new frontiers in physically-reasonable world
modeling and its applications.
Aether is a unified framework that combines 4D dynamic reconstruction, action-conditioned video prediction, and goal-conditioned visual planning, achieving zero-shot generalization and superior reconstruction without real-world data.
4D dynamic reconstruction action-conditioned video prediction goal-conditioned visual planning task-interleaved feature learning synthetic-to-real generalization zero-shot generalization intrinsic geometric modeling geometry-informed action space autonomous trajectory planning