RayGauss: Volumetric Gaussian-Based Ray Casting for Photorealistic Novel
  View Synthesis
Differentiable volumetric rendering-based methods made significant progress
in novel view synthesis. On one hand, innovative methods have replaced the
Neural Radiance Fields (NeRF) network with locally parameterized structures,
enabling high-quality renderings in a reasonable time. On the other hand,
approaches have used differentiable splatting instead of NeRF's ray casting to
optimize radiance fields rapidly using Gaussian kernels, allowing for fine
adaptation to the scene. However, differentiable ray casting of irregularly
spaced kernels has been scarcely explored, while splatting, despite enabling
fast rendering times, is susceptible to clearly visible artifacts.
  Our work closes this gap by providing a physically consistent formulation of
the emitted radiance c and density {\sigma}, decomposed with Gaussian functions
associated with Spherical Gaussians/Harmonics for all-frequency colorimetric
representation. We also introduce a method enabling differentiable ray casting
of irregularly distributed Gaussians using an algorithm that integrates
radiance fields slab by slab and leverages a BVH structure. This allows our
approach to finely adapt to the scene while avoiding splatting artifacts. As a
result, we achieve superior rendering quality compared to the state-of-the-art
while maintaining reasonable training times and achieving inference speeds of
25 FPS on the Blender dataset. Project page with videos and code:
https://raygauss.github.io/
66b422ea3a88f464e26a50b9
Hugo Blanc
claimed_verified
2024-08-08T13:08:17.757Z
66b4c094aa581aef1f61199b
/avatars/ea0665b19091cb84d644da088b13d294.svg
Hugo Blanc
user
hublbl
66b422ea3a88f464e26a50ba
Jean-Emmanuel Deschaud
66b422ea3a88f464e26a50bb
Alexis Paljic
RayGauss: Volumetric Gaussian-Based Ray Casting for Photorealistic Novel
  View Synthesis
Differentiable volumetric rendering-based methods made significant progress
in novel view synthesis. On one hand, innovative methods have replaced the
Neural Radiance Fields (NeRF) network with locally parameterized structures,
enabling high-quality renderings in a reasonable time. On the other hand,
approaches have used differentiable splatting instead of NeRF's ray casting to
optimize radiance fields rapidly using Gaussian kernels, allowing for fine
adaptation to the scene. However, differentiable ray casting of irregularly
spaced kernels has been scarcely explored, while splatting, despite enabling
fast rendering times, is susceptible to clearly visible artifacts.
  Our work closes this gap by providing a physically consistent formulation of
the emitted radiance c and density {\sigma}, decomposed with Gaussian functions
associated with Spherical Gaussians/Harmonics for all-frequency colorimetric
representation. We also introduce a method enabling differentiable ray casting
of irregularly distributed Gaussians using an algorithm that integrates
radiance fields slab by slab and leverages a BVH structure. This allows our
approach to finely adapt to the scene while avoiding splatting artifacts. As a
result, we achieve superior rendering quality compared to the state-of-the-art
while maintaining reasonable training times and achieving inference speeds of
25 FPS on the Blender dataset. Project page with videos and code:
https://raygauss.github.io/
The work introduces a physically consistent volumetric rendering method using differentiable ray casting of irregularly distributed Gaussians, achieving high-quality rendering with fast inference.
Neural Radiance Fields (NeRF) locally parameterized structures differentiable splatting Gaussian kernels differentiable ray casting Gaussian functions Spherical Gaussians/Harmonics BVH structure rendering quality Blender dataset