LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation
3D immersive scene generation is a challenging yet critical task in computer
vision and graphics. A desired virtual 3D scene should 1) exhibit
omnidirectional view consistency, and 2) allow for free exploration in complex
scene hierarchies. Existing methods either rely on successive scene expansion
via inpainting or employ panorama representation to represent large FOV scene
environments. However, the generated scene suffers from semantic drift during
expansion and is unable to handle occlusion among scene hierarchies. To tackle
these challenges, we introduce LayerPano3D, a novel framework for full-view,
explorable panoramic 3D scene generation from a single text prompt. Our key
insight is to decompose a reference 2D panorama into multiple layers at
different depth levels, where each layer reveals the unseen space from the
reference views via diffusion prior. LayerPano3D comprises multiple dedicated
designs: 1) we introduce a novel text-guided anchor view synthesis pipeline for
high-quality, consistent panorama generation. 2) We pioneer the Layered 3D
Panorama as underlying representation to manage complex scene hierarchies and
lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes
with unconstrained viewing paths. Extensive experiments demonstrate that our
framework generates state-of-the-art 3D panoramic scene in both full view
consistency and immersive exploratory experience. We believe that LayerPano3D
holds promise for advancing 3D panoramic scene creation with numerous
applications.
66cbfc93888e99a28d4724c4
Shuai Yang
claimed_verified
2025-06-06T07:43:41.997Z
6373037cd3ba6bd3f9bc32fa
https://cdn-avatars.huggingface.co/v1/production/uploads/6373037cd3ba6bd3f9bc32fa/dIL4AIcL24QL_Rp4B9ewS.png
Shuai Yang
user
ysmikey
66cbfc93888e99a28d4724c5
Jing Tan
claimed_verified
2024-08-27T07:42:42.514Z
65367c40061949598892dbdc
/avatars/4baf27263841471cbd5f629a8b99424d.svg
Jing Tan
user
jingtan
66cbfc93888e99a28d4724c6
Mengchen Zhang
claimed_verified
2024-12-11T09:22:08.873Z
64de20c5808492ba6e65d124
https://cdn-avatars.huggingface.co/v1/production/uploads/64de20c5808492ba6e65d124/58IX_TI5vJw73qS1knw56.jpeg
Zhang Mengchen
user
Dubhe-zmc
66cbfc93888e99a28d4724c7
Tong Wu
claimed_verified
2024-12-11T09:22:11.294Z
6310929fda0aaad2f70397a1
https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
Tong Wu
user
wutong16
66cbfc93888e99a28d4724c8
Yixuan Li
66cbfc93888e99a28d4724c9
Gordon Wetzstein
66cbfc93888e99a28d4724ca
Ziwei Liu
66cbfc93888e99a28d4724cb
Dahua Lin
LayerPano3D: Layered 3D Panorama for Hyper-Immersive Scene Generation
3D immersive scene generation is a challenging yet critical task in computer
vision and graphics. A desired virtual 3D scene should 1) exhibit
omnidirectional view consistency, and 2) allow for free exploration in complex
scene hierarchies. Existing methods either rely on successive scene expansion
via inpainting or employ panorama representation to represent large FOV scene
environments. However, the generated scene suffers from semantic drift during
expansion and is unable to handle occlusion among scene hierarchies. To tackle
these challenges, we introduce LayerPano3D, a novel framework for full-view,
explorable panoramic 3D scene generation from a single text prompt. Our key
insight is to decompose a reference 2D panorama into multiple layers at
different depth levels, where each layer reveals the unseen space from the
reference views via diffusion prior. LayerPano3D comprises multiple dedicated
designs: 1) we introduce a novel text-guided anchor view synthesis pipeline for
high-quality, consistent panorama generation. 2) We pioneer the Layered 3D
Panorama as underlying representation to manage complex scene hierarchies and
lift it into 3D Gaussians to splat detailed 360-degree omnidirectional scenes
with unconstrained viewing paths. Extensive experiments demonstrate that our
framework generates state-of-the-art 3D panoramic scene in both full view
consistency and immersive exploratory experience. We believe that LayerPano3D
holds promise for advancing 3D panoramic scene creation with numerous
applications.
LayerPano3D generates high-quality, full-view consistent, and immersive 3D panoramic scenes from text using layered diffusion priors and Gaussians.
omnidirectional view consistency panorama representation scene inpainting semantic drift diffusion prior text-guided anchor view synthesis Layered 3D Panorama 3D Gaussians full-view consistency immersive exploratory experience