RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent
  Geometry and Texture
The techniques for 3D indoor scene capturing are widely used, but the meshes
produced leave much to be desired. In this paper, we propose "RoomDreamer",
which leverages powerful natural language to synthesize a new room with a
different style. Unlike existing image synthesis methods, our work addresses
the challenge of synthesizing both geometry and texture aligned to the input
scene structure and prompt simultaneously. The key insight is that a scene
should be treated as a whole, taking into account both scene texture and
geometry. The proposed framework consists of two significant components:
Geometry Guided Diffusion and Mesh Optimization. Geometry Guided Diffusion for
3D Scene guarantees the consistency of the scene style by applying the 2D prior
to the entire scene simultaneously. Mesh Optimization improves the geometry and
texture jointly and eliminates the artifacts in the scanned scene. To validate
the proposed method, real indoor scenes scanned with smartphones are used for
extensive experiments, through which the effectiveness of our method is
demonstrated.
646abab435c7a57f936bee50
Liangchen Song
admin_assigned
2023-05-23T09:46:37.283Z
63d9ea2aaefaa7352587151c
/avatars/2f02ee1b2a0018c74ef1d47883782bd5.svg
Liangchen Song
user
lsongx
646abab435c7a57f936bee51
Liangliang Cao
646abab435c7a57f936bee52
Hongyu Xu
646abab435c7a57f936bee53
Kai Kang
646abab435c7a57f936bee54
Feng Tang
646abab435c7a57f936bee55
Junsong Yuan
646abab435c7a57f936bee56
Yang Zhao
RoomDreamer: Text-Driven 3D Indoor Scene Synthesis with Coherent
  Geometry and Texture
The techniques for 3D indoor scene capturing are widely used, but the meshes
produced leave much to be desired. In this paper, we propose "RoomDreamer",
which leverages powerful natural language to synthesize a new room with a
different style. Unlike existing image synthesis methods, our work addresses
the challenge of synthesizing both geometry and texture aligned to the input
scene structure and prompt simultaneously. The key insight is that a scene
should be treated as a whole, taking into account both scene texture and
geometry. The proposed framework consists of two significant components:
Geometry Guided Diffusion and Mesh Optimization. Geometry Guided Diffusion for
3D Scene guarantees the consistency of the scene style by applying the 2D prior
to the entire scene simultaneously. Mesh Optimization improves the geometry and
texture jointly and eliminates the artifacts in the scanned scene. To validate
the proposed method, real indoor scenes scanned with smartphones are used for
extensive experiments, through which the effectiveness of our method is
demonstrated.
RoomDreamer synthesizes 3D indoor scenes with desired styles by aligning geometry and texture using Geometry Guided Diffusion and Mesh Optimization.
Geometry Guided Diffusion Mesh Optimization 3D indoor scene synthesis scene structure prompt scene texture scene geometry artifacts smartphone scanning