GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
Despite recent advances in text-to-3D generative methods, there is a notable
absence of reliable evaluation metrics. Existing metrics usually focus on a
single criterion each, such as how well the asset aligned with the input text.
These metrics lack the flexibility to generalize to different evaluation
criteria and might not align well with human preferences. Conducting user
preference studies is an alternative that offers both adaptability and
human-aligned results. User studies, however, can be very expensive to scale.
This paper presents an automatic, versatile, and human-aligned evaluation
metric for text-to-3D generative models. To this end, we first develop a prompt
generator using GPT-4V to generate evaluating prompts, which serve as input to
compare text-to-3D models. We further design a method instructing GPT-4V to
compare two 3D assets according to user-defined criteria. Finally, we use these
pairwise comparison results to assign these models Elo ratings. Experimental
results suggest our metric strongly align with human preference across
different evaluation criteria.
659cbaa5e9095ff98daf4b2b
Tong Wu
claimed_verified
2024-12-11T09:24:52.104Z
6310929fda0aaad2f70397a1
https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
Tong Wu
user
wutong16
659cbaa5e9095ff98daf4b2c
Guandao Yang
admin_assigned
2024-01-09T08:10:36.236Z
64547ca006728ff79a38e7a5
https://cdn-avatars.huggingface.co/v1/production/uploads/64547ca006728ff79a38e7a5/xfTVQXMSfRv-IbljFpswQ.jpeg
Guandao Yang
user
guandao
659cbaa5e9095ff98daf4b2d
Zhibing Li
admin_assigned
2024-01-09T08:10:42.628Z
632a80706813868fa4a649e3
https://cdn-avatars.huggingface.co/v1/production/uploads/632a80706813868fa4a649e3/MbTsAYGadNwS3-G5vxEnm.jpeg
Zhibing LI
user
lizb6626
659cbaa5e9095ff98daf4b2e
Kai Zhang
admin_assigned
2024-01-09T08:11:28.479Z
63e0a50242591dda0b9dca5c
https://cdn-avatars.huggingface.co/v1/production/uploads/63e0a50242591dda0b9dca5c/c7cBPEBWQDFYimfGnO_SI.png
Kai Zhang
user
drogozhang
659cbaa5e9095ff98daf4b2f
Ziwei Liu
admin_assigned
2024-01-09T08:12:17.820Z
62ab1ac1d48b4d8b048a3473
https://cdn-avatars.huggingface.co/v1/production/uploads/1656826685333-62ab1ac1d48b4d8b048a3473.png
Ziwei Liu
user
liuziwei7
659cbaa5e9095ff98daf4b30
Leonidas Guibas
659cbaa5e9095ff98daf4b31
Dahua Lin
admin_assigned
2024-01-09T08:12:34.229Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
659cbaa5e9095ff98daf4b32
Gordon Wetzstein
GPT-4V(ision) is a Human-Aligned Evaluator for Text-to-3D Generation
Despite recent advances in text-to-3D generative methods, there is a notable
absence of reliable evaluation metrics. Existing metrics usually focus on a
single criterion each, such as how well the asset aligned with the input text.
These metrics lack the flexibility to generalize to different evaluation
criteria and might not align well with human preferences. Conducting user
preference studies is an alternative that offers both adaptability and
human-aligned results. User studies, however, can be very expensive to scale.
This paper presents an automatic, versatile, and human-aligned evaluation
metric for text-to-3D generative models. To this end, we first develop a prompt
generator using GPT-4V to generate evaluating prompts, which serve as input to
compare text-to-3D models. We further design a method instructing GPT-4V to
compare two 3D assets according to user-defined criteria. Finally, we use these
pairwise comparison results to assign these models Elo ratings. Experimental
results suggest our metric strongly align with human preference across
different evaluation criteria.
An automatic evaluation metric for text-to-3D generative models uses GPT-4V to generate prompts and perform pairwise comparisons for assigning Elo ratings, aligning with human preferences.
GPT-4V prompt generator pairwise comparison Elo ratings