DreamDPO: Aligning Text-to-3D Generation with Human Preferences via
  Direct Preference Optimization
Text-to-3D generation automates 3D content creation from textual
descriptions, which offers transformative potential across various fields.
However, existing methods often struggle to align generated content with human
preferences, limiting their applicability and flexibility. To address these
limitations, in this paper, we propose DreamDPO, an optimization-based
framework that integrates human preferences into the 3D generation process,
through direct preference optimization. Practically, DreamDPO first constructs
pairwise examples, then compare their alignment with human preferences using
reward or large multimodal models, and lastly optimizes the 3D representation
with a preference-driven loss function. By leveraging pairwise comparison to
reflect preferences, DreamDPO reduces reliance on precise pointwise quality
evaluations while enabling fine-grained controllability through
preference-guided optimization. Experiments demonstrate that DreamDPO achieves
competitive results, and provides higher-quality and more controllable 3D
content compared to existing methods. The code and models will be open-sourced.
67aafd90141fac22732a79b3
Zhenglin Zhou
admin_assigned
2025-02-11T15:53:27.764Z
6425318d175bd2952281065e
/avatars/37deb6ceb1552dece43a1c8c13c1c871.svg
ZhenglinZhou
user
zhenglin
67aafd90141fac22732a79b4
Xiaobo Xia
admin_assigned
2025-02-11T15:53:52.391Z
66c48f944511077b9ff5ce9d
/avatars/a4977246bc951e9da0cb2301bedd8249.svg
Xiaobo Xia
user
XiaoboXia1997
67aafd90141fac22732a79b5
Fan Ma
67aafd90141fac22732a79b6
Hehe Fan
admin_assigned
2025-02-11T15:54:14.222Z
64ad04020fb9b20dbabbd30e
/avatars/a6bae4a3a4bcd6b54c33860fe14c7923.svg
Hehe Fan
user
hehefan
67aafd90141fac22732a79b7
Yi Yang
67aafd90141fac22732a79b8
Tat-Seng Chua
admin_assigned
2025-02-11T15:54:04.147Z
6570ae84c4993b8fb96f41a8
/avatars/21f7d79d46ac4df0ecff8eca7678b33f.svg
Tat-Seng Chua
user
chuats
DreamDPO: Aligning Text-to-3D Generation with Human Preferences via
  Direct Preference Optimization
Text-to-3D generation automates 3D content creation from textual
descriptions, which offers transformative potential across various fields.
However, existing methods often struggle to align generated content with human
preferences, limiting their applicability and flexibility. To address these
limitations, in this paper, we propose DreamDPO, an optimization-based
framework that integrates human preferences into the 3D generation process,
through direct preference optimization. Practically, DreamDPO first constructs
pairwise examples, then compare their alignment with human preferences using
reward or large multimodal models, and lastly optimizes the 3D representation
with a preference-driven loss function. By leveraging pairwise comparison to
reflect preferences, DreamDPO reduces reliance on precise pointwise quality
evaluations while enabling fine-grained controllability through
preference-guided optimization. Experiments demonstrate that DreamDPO achieves
competitive results, and provides higher-quality and more controllable 3D
content compared to existing methods. The code and models will be open-sourced.
DreamDPO uses preference optimization through pairwise comparisons and reward models to generate high-quality, controllable 3D content from textual descriptions.
text-to-3D generation DreamDPO optimization-based framework direct preference optimization pairwise examples reward models large multimodal models preference-driven loss function preference optimization controllable 3D content