GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D
  Generation
While 3D content generation has advanced significantly, existing methods
still face challenges with input formats, latent space design, and output
representations. This paper introduces a novel 3D generation framework that
addresses these challenges, offering scalable, high-quality 3D generation with
an interactive Point Cloud-structured Latent space. Our framework employs a
Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal)
renderings as input, using a unique latent space design that preserves 3D shape
information, and incorporates a cascaded latent diffusion model for improved
shape-texture disentanglement. The proposed method, GaussianAnything, supports
multi-modal conditional 3D generation, allowing for point cloud, caption, and
single/multi-view image inputs. Notably, the newly proposed latent space
naturally enables geometry-texture disentanglement, thus allowing 3D-aware
editing. Experimental results demonstrate the effectiveness of our approach on
multiple datasets, outperforming existing methods in both text- and
image-conditioned 3D generation.
673aa0f7c36fe6d2f0a1a9a9
Yushi Lan
claimed_verified
2024-11-18T07:47:19.142Z
64b55f62e6b0c53f97e64034
/avatars/d5fc53de37d4fb0b5095a5fb21b43503.svg
LAN YUSHI
user
yslan
673aa0f7c36fe6d2f0a1a9aa
Shangchen Zhou
admin_assigned
2024-11-18T07:49:40.488Z
62e57662ae9d3f10acbb1b1b
https://cdn-avatars.huggingface.co/v1/production/uploads/62e57662ae9d3f10acbb1b1b/lg58jdbNyv6LGH2LFnZDF.png
Shangchen Zhou
user
sczhou
673aa0f7c36fe6d2f0a1a9ab
Zhaoyang Lyu
admin_assigned
2024-11-18T07:49:45.706Z
63f2ec797ddf724fbcc75aee
/avatars/e93432ad11da703d46fe5e594d69f8c0.svg
Zhaoyang Lyu
user
ZhaoyangLyu
673aa0f7c36fe6d2f0a1a9ac
Fangzhou Hong
admin_assigned
2024-11-18T07:49:50.863Z
623c530013a63ea865f96c8e
/avatars/164455a1a94f92b71733fc778c21bd89.svg
Fangzhou Hong
user
hongfz16
673aa0f7c36fe6d2f0a1a9ad
Shuai Yang
673aa0f7c36fe6d2f0a1a9ae
Bo Dai
admin_assigned
2024-11-18T07:50:09.291Z
665e2bd0fbaa279db369b488
/avatars/8cbc18d17d2813617976dbe709235799.svg
Bo Dai
user
doubling
673aa0f7c36fe6d2f0a1a9af
Xingang Pan
admin_assigned
2024-11-18T07:50:35.048Z
646758d6a9b4610868a138ff
https://cdn-avatars.huggingface.co/v1/production/uploads/646758d6a9b4610868a138ff/fNBlK5g4g7vlPAR1bQimZ.png
Xingang Pan
user
Xingang-Pan
673aa0f7c36fe6d2f0a1a9b0
Chen Change Loy
GaussianAnything: Interactive Point Cloud Latent Diffusion for 3D
  Generation
While 3D content generation has advanced significantly, existing methods
still face challenges with input formats, latent space design, and output
representations. This paper introduces a novel 3D generation framework that
addresses these challenges, offering scalable, high-quality 3D generation with
an interactive Point Cloud-structured Latent space. Our framework employs a
Variational Autoencoder (VAE) with multi-view posed RGB-D(epth)-N(ormal)
renderings as input, using a unique latent space design that preserves 3D shape
information, and incorporates a cascaded latent diffusion model for improved
shape-texture disentanglement. The proposed method, GaussianAnything, supports
multi-modal conditional 3D generation, allowing for point cloud, caption, and
single/multi-view image inputs. Notably, the newly proposed latent space
naturally enables geometry-texture disentanglement, thus allowing 3D-aware
editing. Experimental results demonstrate the effectiveness of our approach on
multiple datasets, outperforming existing methods in both text- and
image-conditioned 3D generation.
A novel 3D generation framework using a VAE and cascaded latent diffusion model in a Point Cloud-structured Latent space achieves high-quality 3D generation with multi-modal inputs and disentangled shape-texture editing.
Variational Autoencoder latent space RGB-D multi-view latent diffusion model shape-texture disentanglement Point Cloud-structured Latent space GaussianAnything 3D-aware editing