SyncDreamer: Generating Multiview-consistent Images from a Single-view
  Image
In this paper, we present a novel diffusion model called that generates
multiview-consistent images from a single-view image. Using pretrained
large-scale 2D diffusion models, recent work Zero123 demonstrates the ability
to generate plausible novel views from a single-view image of an object.
However, maintaining consistency in geometry and colors for the generated
images remains a challenge. To address this issue, we propose a synchronized
multiview diffusion model that models the joint probability distribution of
multiview images, enabling the generation of multiview-consistent images in a
single reverse process. SyncDreamer synchronizes the intermediate states of all
the generated images at every step of the reverse process through a 3D-aware
feature attention mechanism that correlates the corresponding features across
different views. Experiments show that SyncDreamer generates images with high
consistency across different views, thus making it well-suited for various 3D
generation tasks such as novel-view-synthesis, text-to-3D, and image-to-3D.
64faa342d75293f417c44529
Yuan Liu
claimed_verified
2024-05-06T07:05:44.865Z
63f5dc824b831cc179b90e8f
/avatars/2c2d22c6b605a9c412a9c37960efeeb2.svg
Yuan Liu
user
liuyuan-pal
64faa342d75293f417c4452a
Cheng Lin
64faa342d75293f417c4452b
Zijiao Zeng
64faa342d75293f417c4452c
Xiaoxiao Long
64faa342d75293f417c4452d
Lingjie Liu
64faa342d75293f417c4452e
Taku Komura
64faa342d75293f417c4452f
Wenping Wang
SyncDreamer: Generating Multiview-consistent Images from a Single-view
  Image
In this paper, we present a novel diffusion model called that generates
multiview-consistent images from a single-view image. Using pretrained
large-scale 2D diffusion models, recent work Zero123 demonstrates the ability
to generate plausible novel views from a single-view image of an object.
However, maintaining consistency in geometry and colors for the generated
images remains a challenge. To address this issue, we propose a synchronized
multiview diffusion model that models the joint probability distribution of
multiview images, enabling the generation of multiview-consistent images in a
single reverse process. SyncDreamer synchronizes the intermediate states of all
the generated images at every step of the reverse process through a 3D-aware
feature attention mechanism that correlates the corresponding features across
different views. Experiments show that SyncDreamer generates images with high
consistency across different views, thus making it well-suited for various 3D
generation tasks such as novel-view-synthesis, text-to-3D, and image-to-3D.
SyncDreamer generates multiview-consistent images using a synchronized diffusion model and 3D-aware feature attention mechanism, improving the quality of 3D generation tasks.
diffusion model multiview-consistent images pretrained large-scale 2D diffusion models synchronized multiview diffusion model joint probability distribution 3D-aware feature attention mechanism multiview images reverse process novel-view-synthesis text-to-3D image-to-3D