SPATIALGEN: Layout-guided 3D Indoor Scene Generation
Creating high-fidelity 3D models of indoor environments is essential for
applications in design, virtual reality, and robotics. However, manual 3D
modeling remains time-consuming and labor-intensive. While recent advances in
generative AI have enabled automated scene synthesis, existing methods often
face challenges in balancing visual quality, diversity, semantic consistency,
and user control. A major bottleneck is the lack of a large-scale, high-quality
dataset tailored to this task. To address this gap, we introduce a
comprehensive synthetic dataset, featuring 12,328 structured annotated scenes
with 57,440 rooms, and 4.7M photorealistic 2D renderings. Leveraging this
dataset, we present SpatialGen, a novel multi-view multi-modal diffusion model
that generates realistic and semantically consistent 3D indoor scenes. Given a
3D layout and a reference image (derived from a text prompt), our model
synthesizes appearance (color image), geometry (scene coordinate map), and
semantic (semantic segmentation map) from arbitrary viewpoints, while
preserving spatial consistency across modalities. SpatialGen consistently
generates superior results to previous methods in our experiments. We are
open-sourcing our data and models to empower the community and advance the
field of indoor scene understanding and generation.
68d0c1a98adc5cd018d15b1f
Chuan Fang
68d0c1a98adc5cd018d15b20
Heng Li
68d0c1a98adc5cd018d15b21
Yixun Liang
68d0c1a98adc5cd018d15b22
Jia Zheng
claimed_verified
2025-09-22T10:31:43.211Z
6437c0ead38ce48bdd4b0067
https://cdn-avatars.huggingface.co/v1/production/uploads/6437c0ead38ce48bdd4b0067/9HdcjbD0ugiPypLDtQl8E.png
Jia Zheng
user
bertjiazheng
68d0c1a98adc5cd018d15b23
Yongsen Mao
68d0c1a98adc5cd018d15b24
Yuan Liu
68d0c1a98adc5cd018d15b25
Rui Tang
68d0c1a98adc5cd018d15b26
Zihan Zhou
68d0c1a98adc5cd018d15b27
Ping Tan
SPATIALGEN: Layout-guided 3D Indoor Scene Generation
Creating high-fidelity 3D models of indoor environments is essential for
applications in design, virtual reality, and robotics. However, manual 3D
modeling remains time-consuming and labor-intensive. While recent advances in
generative AI have enabled automated scene synthesis, existing methods often
face challenges in balancing visual quality, diversity, semantic consistency,
and user control. A major bottleneck is the lack of a large-scale, high-quality
dataset tailored to this task. To address this gap, we introduce a
comprehensive synthetic dataset, featuring 12,328 structured annotated scenes
with 57,440 rooms, and 4.7M photorealistic 2D renderings. Leveraging this
dataset, we present SpatialGen, a novel multi-view multi-modal diffusion model
that generates realistic and semantically consistent 3D indoor scenes. Given a
3D layout and a reference image (derived from a text prompt), our model
synthesizes appearance (color image), geometry (scene coordinate map), and
semantic (semantic segmentation map) from arbitrary viewpoints, while
preserving spatial consistency across modalities. SpatialGen consistently
generates superior results to previous methods in our experiments. We are
open-sourcing our data and models to empower the community and advance the
field of indoor scene understanding and generation.
SpatialGen, a multi-view multi-modal diffusion model, generates realistic and semantically consistent 3D indoor scenes using a large synthetic dataset, outperforming previous methods.
diffusion model 3D indoor scenes synthetic dataset 3D layout reference image text prompt appearance geometry semantic segmentation map spatial consistency multi-view multi-modal