3D Arena: An Open Platform for Generative 3D Evaluation
Evaluating Generative 3D models remains challenging due to misalignment
between automated metrics and human perception of quality. Current benchmarks
rely on image-based metrics that ignore 3D structure or geometric measures that
fail to capture perceptual appeal and real-world utility. To address this gap,
we present 3D Arena, an open platform for evaluating image-to-3D generation
models through large-scale human preference collection using pairwise
comparisons.
  Since launching in June 2024, the platform has collected 123,243 votes from
8,096 users across 19 state-of-the-art models, establishing the largest human
preference evaluation for Generative 3D. We contribute the iso3d dataset of 100
evaluation prompts and demonstrate quality control achieving 99.75% user
authenticity through statistical fraud detection. Our ELO-based ranking system
provides reliable model assessment, with the platform becoming an established
evaluation resource.
  Through analysis of this preference data, we present insights into human
preference patterns. Our findings reveal preferences for visual presentation
features, with Gaussian splat outputs achieving a 16.6 ELO advantage over
meshes and textured models receiving a 144.1 ELO advantage over untextured
models. We provide recommendations for improving evaluation methods, including
multi-criteria assessment, task-oriented evaluation, and format-aware
comparison. The platform's community engagement establishes 3D Arena as a
benchmark for the field while advancing understanding of human-centered
evaluation in Generative 3D.
685a19100e4ad7e219758595
Dylan Ebert
claimed_verified
2025-06-24T16:25:14.250Z
624b4a964056e2a6914a05c5
https://cdn-avatars.huggingface.co/v1/production/uploads/1672164046414-624b4a964056e2a6914a05c5.png
Dylan Ebert
user
dylanebert
3D Arena: An Open Platform for Generative 3D Evaluation
Evaluating Generative 3D models remains challenging due to misalignment
between automated metrics and human perception of quality. Current benchmarks
rely on image-based metrics that ignore 3D structure or geometric measures that
fail to capture perceptual appeal and real-world utility. To address this gap,
we present 3D Arena, an open platform for evaluating image-to-3D generation
models through large-scale human preference collection using pairwise
comparisons.
  Since launching in June 2024, the platform has collected 123,243 votes from
8,096 users across 19 state-of-the-art models, establishing the largest human
preference evaluation for Generative 3D. We contribute the iso3d dataset of 100
evaluation prompts and demonstrate quality control achieving 99.75% user
authenticity through statistical fraud detection. Our ELO-based ranking system
provides reliable model assessment, with the platform becoming an established
evaluation resource.
  Through analysis of this preference data, we present insights into human
preference patterns. Our findings reveal preferences for visual presentation
features, with Gaussian splat outputs achieving a 16.6 ELO advantage over
meshes and textured models receiving a 144.1 ELO advantage over untextured
models. We provide recommendations for improving evaluation methods, including
multi-criteria assessment, task-oriented evaluation, and format-aware
comparison. The platform's community engagement establishes 3D Arena as a
benchmark for the field while advancing understanding of human-centered
evaluation in Generative 3D.
3D Arena evaluates generative 3D models using human preferences, revealing insights into visual and textural features' impact on quality.
3D Arena human preference collection image-to-3D generation pairwise comparisons iso3d dataset quality control statistical fraud detection ELO-based ranking system Gaussian splat mesh textured models untextured models multi-criteria assessment task-oriented evaluation format-aware comparison