HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D
Recent progress in single-image 3D generation highlights the importance of
multi-view coherency, leveraging 3D priors from large-scale diffusion models
pretrained on Internet-scale images. However, the aspect of novel-view
diversity remains underexplored within the research landscape due to the
ambiguity in converting a 2D image into 3D content, where numerous potential
shapes can emerge. Here, we aim to address this research gap by simultaneously
addressing both consistency and diversity. Yet, striking a balance between
these two aspects poses a considerable challenge due to their inherent
trade-offs. This work introduces HarmonyView, a simple yet effective diffusion
sampling technique adept at decomposing two intricate aspects in single-image
3D generation: consistency and diversity. This approach paves the way for a
more nuanced exploration of the two critical dimensions within the sampling
process. Moreover, we propose a new evaluation metric based on CLIP image and
text encoders to comprehensively assess the diversity of the generated views,
which closely aligns with human evaluators' judgments. In experiments,
HarmonyView achieves a harmonious balance, demonstrating a win-win scenario in
both consistency and diversity.
658b900b5c6fb5d5e321d891
Sangmin Woo
claimed_verified
2024-06-30T20:59:00.659Z
64a3d9f467abe0e3d360151d
/avatars/d0d8772cfdb6e8571f456ef31c7871f1.svg
Sangmin Woo
user
sangminwoo
658b900b5c6fb5d5e321d892
Byeongjun Park
claimed_verified
2023-12-27T11:05:48.777Z
653929a66da48e0d21e65e17
/avatars/e34ae3411d689b4280ff34c1b680f283.svg
Byeongjun Park
user
byeongjun-park
658b900b5c6fb5d5e321d893
Hyojun Go
claimed_verified
2024-01-15T07:42:14.585Z
649f65a4ca03a1a35e3dac14
/avatars/b0dcd8ad795b1e666ee247b2ac024d53.svg
Hyojun GO
user
HJGO
658b900b5c6fb5d5e321d894
Jin-Young Kim
658b900b5c6fb5d5e321d895
Changick Kim
HarmonyView: Harmonizing Consistency and Diversity in One-Image-to-3D
Recent progress in single-image 3D generation highlights the importance of
multi-view coherency, leveraging 3D priors from large-scale diffusion models
pretrained on Internet-scale images. However, the aspect of novel-view
diversity remains underexplored within the research landscape due to the
ambiguity in converting a 2D image into 3D content, where numerous potential
shapes can emerge. Here, we aim to address this research gap by simultaneously
addressing both consistency and diversity. Yet, striking a balance between
these two aspects poses a considerable challenge due to their inherent
trade-offs. This work introduces HarmonyView, a simple yet effective diffusion
sampling technique adept at decomposing two intricate aspects in single-image
3D generation: consistency and diversity. This approach paves the way for a
more nuanced exploration of the two critical dimensions within the sampling
process. Moreover, we propose a new evaluation metric based on CLIP image and
text encoders to comprehensively assess the diversity of the generated views,
which closely aligns with human evaluators' judgments. In experiments,
HarmonyView achieves a harmonious balance, demonstrating a win-win scenario in
both consistency and diversity.
HarmonyView, a diffusion sampling technique, addresses consistency and diversity in single-image 3D generation and introduces a CLIP-based evaluation metric for diversity.
multi-view coherency diffusion models novel-view diversity single-image 3D generation consistency diversity diffusion sampling technique CLIP image and text encoders