ProNeRF: Learning Efficient Projection-Aware Ray Sampling for
  Fine-Grained Implicit Neural Radiance Fields
Recent advances in neural rendering have shown that, albeit slow, implicit
compact models can learn a scene's geometries and view-dependent appearances
from multiple views. To maintain such a small memory footprint but achieve
faster inference times, recent works have adopted `sampler' networks that
adaptively sample a small subset of points along each ray in the implicit
neural radiance fields. Although these methods achieve up to a 10times
reduction in rendering time, they still suffer from considerable quality
degradation compared to the vanilla NeRF. In contrast, we propose ProNeRF,
which provides an optimal trade-off between memory footprint (similar to NeRF),
speed (faster than HyperReel), and quality (better than K-Planes). ProNeRF is
equipped with a novel projection-aware sampling (PAS) network together with a
new training strategy for ray exploration and exploitation, allowing for
efficient fine-grained particle sampling. Our ProNeRF yields state-of-the-art
metrics, being 15-23x faster with 0.65dB higher PSNR than NeRF and yielding
0.95dB higher PSNR than the best published sampler-based method, HyperReel. Our
exploration and exploitation training strategy allows ProNeRF to learn the full
scenes' color and density distributions while also learning efficient ray
sampling focused on the highest-density regions. We provide extensive
experimental results that support the effectiveness of our method on the widely
adopted forward-facing and 360 datasets, LLFF and Blender, respectively.
657a75bff10a984150a495b6
Juan Luis Gonzalez Bello
claimed_verified
2023-12-18T08:48:22.582Z
657ff1721815b29c9a1e485c
/avatars/99143e93514f90c9dea282f182185941.svg
Juan Luis Gonzalez Bello
user
JuanLuisGB
657a75bff10a984150a495b7
Minh-Quan Viet Bui
claimed_verified
2023-12-22T14:29:01.484Z
65859653c8b8bd66f7c5a651
/avatars/84825a8224350dae85784534e9c59424.svg
Quan Bui
user
quan5609
657a75bff10a984150a495b8
Munchurl Kim
ProNeRF: Learning Efficient Projection-Aware Ray Sampling for
  Fine-Grained Implicit Neural Radiance Fields
Recent advances in neural rendering have shown that, albeit slow, implicit
compact models can learn a scene's geometries and view-dependent appearances
from multiple views. To maintain such a small memory footprint but achieve
faster inference times, recent works have adopted `sampler' networks that
adaptively sample a small subset of points along each ray in the implicit
neural radiance fields. Although these methods achieve up to a 10times
reduction in rendering time, they still suffer from considerable quality
degradation compared to the vanilla NeRF. In contrast, we propose ProNeRF,
which provides an optimal trade-off between memory footprint (similar to NeRF),
speed (faster than HyperReel), and quality (better than K-Planes). ProNeRF is
equipped with a novel projection-aware sampling (PAS) network together with a
new training strategy for ray exploration and exploitation, allowing for
efficient fine-grained particle sampling. Our ProNeRF yields state-of-the-art
metrics, being 15-23x faster with 0.65dB higher PSNR than NeRF and yielding
0.95dB higher PSNR than the best published sampler-based method, HyperReel. Our
exploration and exploitation training strategy allows ProNeRF to learn the full
scenes' color and density distributions while also learning efficient ray
sampling focused on the highest-density regions. We provide extensive
experimental results that support the effectiveness of our method on the widely
adopted forward-facing and 360 datasets, LLFF and Blender, respectively.
ProNeRF improves neural rendering by balancing memory usage, speed, and quality through projection-aware sampling and a novel training strategy.
implicit compact models implicit neural radiance fields sampler networks ProNeRF projection-aware sampling ray exploration ray exploitation PSNR NeRF HyperReel K-Planes LLFF Blender