AIM 2024 Sparse Neural Rendering Challenge: Dataset and Benchmark
Recent developments in differentiable and neural rendering have made
impressive breakthroughs in a variety of 2D and 3D tasks, e.g. novel view
synthesis, 3D reconstruction. Typically, differentiable rendering relies on a
dense viewpoint coverage of the scene, such that the geometry can be
disambiguated from appearance observations alone. Several challenges arise when
only a few input views are available, often referred to as sparse or few-shot
neural rendering. As this is an underconstrained problem, most existing
approaches introduce the use of regularisation, together with a diversity of
learnt and hand-crafted priors. A recurring problem in sparse rendering
literature is the lack of an homogeneous, up-to-date, dataset and evaluation
protocol. While high-resolution datasets are standard in dense reconstruction
literature, sparse rendering methods often evaluate with low-resolution images.
Additionally, data splits are inconsistent across different manuscripts, and
testing ground-truth images are often publicly available, which may lead to
over-fitting. In this work, we propose the Sparse Rendering (SpaRe) dataset and
benchmark. We introduce a new dataset that follows the setup of the DTU MVS
dataset. The dataset is composed of 97 new scenes based on synthetic,
high-quality assets. Each scene has up to 64 camera views and 7 lighting
configurations, rendered at 1600x1200 resolution. We release a training split
of 82 scenes to foster generalizable approaches, and provide an online
evaluation platform for the validation and test sets, whose ground-truth images
remain hidden. We propose two different sparse configurations (3 and 9 input
images respectively). This provides a powerful and convenient tool for
reproducible evaluation, and enable researchers easy access to a public
leaderboard with the state-of-the-art performance scores. Available at:
https://sparebenchmark.github.io/
66f449bbf0eb32a95335e521
Michal Nazarczuk
claimed_verified
2024-09-25T18:24:07.881Z
66f44a3df9252e0f50b59fdb
/avatars/4234203c77a0e6f594f3de26bfe8c649.svg
Michal Nazarczuk
user
michaal94
66f449bbf0eb32a95335e522
Thomas Tanay
admin_assigned
2024-09-26T09:41:51.930Z
65de232d398a8e9771c63c60
https://cdn-avatars.huggingface.co/v1/production/uploads/65de232d398a8e9771c63c60/mw0szrwxxyAwq8fmAht7u.jpeg
Thomas Tanay
user
thomas-tanay
66f449bbf0eb32a95335e523
Sibi Catley-Chandar
admin_assigned
2024-09-26T09:41:57.714Z
66f42a618547c4e49394655c
/avatars/094312c81e64eabf8c97f9cd643e1136.svg
Sibi Catley-Chandar
user
sibicatleychandar
66f449bbf0eb32a95335e524
Richard Shaw
66f449bbf0eb32a95335e525
Radu Timofte
66f449bbf0eb32a95335e526
Eduardo Pérez-Pellitero
AIM 2024 Sparse Neural Rendering Challenge: Dataset and Benchmark
Recent developments in differentiable and neural rendering have made
impressive breakthroughs in a variety of 2D and 3D tasks, e.g. novel view
synthesis, 3D reconstruction. Typically, differentiable rendering relies on a
dense viewpoint coverage of the scene, such that the geometry can be
disambiguated from appearance observations alone. Several challenges arise when
only a few input views are available, often referred to as sparse or few-shot
neural rendering. As this is an underconstrained problem, most existing
approaches introduce the use of regularisation, together with a diversity of
learnt and hand-crafted priors. A recurring problem in sparse rendering
literature is the lack of an homogeneous, up-to-date, dataset and evaluation
protocol. While high-resolution datasets are standard in dense reconstruction
literature, sparse rendering methods often evaluate with low-resolution images.
Additionally, data splits are inconsistent across different manuscripts, and
testing ground-truth images are often publicly available, which may lead to
over-fitting. In this work, we propose the Sparse Rendering (SpaRe) dataset and
benchmark. We introduce a new dataset that follows the setup of the DTU MVS
dataset. The dataset is composed of 97 new scenes based on synthetic,
high-quality assets. Each scene has up to 64 camera views and 7 lighting
configurations, rendered at 1600x1200 resolution. We release a training split
of 82 scenes to foster generalizable approaches, and provide an online
evaluation platform for the validation and test sets, whose ground-truth images
remain hidden. We propose two different sparse configurations (3 and 9 input
images respectively). This provides a powerful and convenient tool for
reproducible evaluation, and enable researchers easy access to a public
leaderboard with the state-of-the-art performance scores. Available at:
https://sparebenchmark.github.io/
A new dataset and benchmark, Sparse Rendering (SpaRe), is introduced for sparse or few-shot neural rendering, providing high-quality synthetic scenes for reproducible evaluation and comparison of performance.
differentiable rendering neural rendering sparse rendering few-shot neural rendering regularization priors dataset evaluation protocol DTU MVS dataset ground-truth images