Reconstructing 4D Spatial Intelligence: A Survey
Reconstructing 4D spatial intelligence from visual observations has long been
a central yet challenging task in computer vision, with broad real-world
applications. These range from entertainment domains like movies, where the
focus is often on reconstructing fundamental visual elements, to embodied AI,
which emphasizes interaction modeling and physical realism. Fueled by rapid
advances in 3D representations and deep learning architectures, the field has
evolved quickly, outpacing the scope of previous surveys. Additionally,
existing surveys rarely offer a comprehensive analysis of the hierarchical
structure of 4D scene reconstruction. To address this gap, we present a new
perspective that organizes existing methods into five progressive levels of 4D
spatial intelligence: (1) Level 1 -- reconstruction of low-level 3D attributes
(e.g., depth, pose, and point maps); (2) Level 2 -- reconstruction of 3D scene
components (e.g., objects, humans, structures); (3) Level 3 -- reconstruction
of 4D dynamic scenes; (4) Level 4 -- modeling of interactions among scene
components; and (5) Level 5 -- incorporation of physical laws and constraints.
We conclude the survey by discussing the key challenges at each level and
highlighting promising directions for advancing toward even richer levels of 4D
spatial intelligence. To track ongoing developments, we maintain an up-to-date
project page: https://github.com/yukangcao/Awesome-4D-Spatial-Intelligence.
68882eddaf872d625c10c5d4
Yukang Cao
68882eddaf872d625c10c5d5
Jiahao Lu
68882eddaf872d625c10c5d6
Zhisheng Huang
68882eddaf872d625c10c5d7
Zhuowei Shen
68882eddaf872d625c10c5d8
Chengfeng Zhao
68882eddaf872d625c10c5d9
Fangzhou Hong
68882eddaf872d625c10c5da
Zhaoxi Chen
68882eddaf872d625c10c5db
Xin Li
68882eddaf872d625c10c5dc
Wenping Wang
68882eddaf872d625c10c5dd
Yuan Liu
68882eddaf872d625c10c5de
Ziwei Liu
Reconstructing 4D Spatial Intelligence: A Survey
Reconstructing 4D spatial intelligence from visual observations has long been
a central yet challenging task in computer vision, with broad real-world
applications. These range from entertainment domains like movies, where the
focus is often on reconstructing fundamental visual elements, to embodied AI,
which emphasizes interaction modeling and physical realism. Fueled by rapid
advances in 3D representations and deep learning architectures, the field has
evolved quickly, outpacing the scope of previous surveys. Additionally,
existing surveys rarely offer a comprehensive analysis of the hierarchical
structure of 4D scene reconstruction. To address this gap, we present a new
perspective that organizes existing methods into five progressive levels of 4D
spatial intelligence: (1) Level 1 -- reconstruction of low-level 3D attributes
(e.g., depth, pose, and point maps); (2) Level 2 -- reconstruction of 3D scene
components (e.g., objects, humans, structures); (3) Level 3 -- reconstruction
of 4D dynamic scenes; (4) Level 4 -- modeling of interactions among scene
components; and (5) Level 5 -- incorporation of physical laws and constraints.
We conclude the survey by discussing the key challenges at each level and
highlighting promising directions for advancing toward even richer levels of 4D
spatial intelligence. To track ongoing developments, we maintain an up-to-date
project page: https://github.com/yukangcao/Awesome-4D-Spatial-Intelligence.
A survey organizes 4D spatial intelligence reconstruction methods into five levels, from low-level 3D attributes to incorporating physical laws, and discusses challenges and future directions.
3D representations deep learning architectures 4D scene reconstruction hierarchical structure 4D dynamic scenes interaction modeling physical laws constraints