NeRF-Casting: Improved View-Dependent Appearance with Consistent
  Reflections
Neural Radiance Fields (NeRFs) typically struggle to reconstruct and render
highly specular objects, whose appearance varies quickly with changes in
viewpoint. Recent works have improved NeRF's ability to render detailed
specular appearance of distant environment illumination, but are unable to
synthesize consistent reflections of closer content. Moreover, these techniques
rely on large computationally-expensive neural networks to model outgoing
radiance, which severely limits optimization and rendering speed. We address
these issues with an approach based on ray tracing: instead of querying an
expensive neural network for the outgoing view-dependent radiance at points
along each camera ray, our model casts reflection rays from these points and
traces them through the NeRF representation to render feature vectors which are
decoded into color using a small inexpensive network. We demonstrate that our
model outperforms prior methods for view synthesis of scenes containing shiny
objects, and that it is the only existing NeRF method that can synthesize
photorealistic specular appearance and reflections in real-world scenes, while
requiring comparable optimization time to current state-of-the-art view
synthesis models.
6650a0f22fbc9f8285e6ffad
Dor Verbin
admin_assigned
2024-05-24T15:16:14.532Z
6315b0fc915d0b806830b424
/avatars/e25a7c8df2364c6e8024ec5c646a2889.svg
Dor Verbin
user
dorverbin
6650a0f22fbc9f8285e6ffae
Pratul P. Srinivasan
6650a0f22fbc9f8285e6ffaf
Peter Hedman
admin_assigned
2024-05-24T15:16:44.507Z
651ec3aee17ef6a94eefc8e7
/avatars/e90ce400ec0d2de8f9fe4365a6e07b4f.svg
Peter Hedman
user
phedman
6650a0f22fbc9f8285e6ffb0
Ben Mildenhall
6650a0f22fbc9f8285e6ffb1
Benjamin Attal
admin_assigned
2024-05-24T15:17:01.177Z
63910c04b73a0f8c02a88b74
/avatars/f099b3f45a683f13537db58a2279bd34.svg
Benjamin Attal
user
battal
6650a0f22fbc9f8285e6ffb2
Richard Szeliski
6650a0f22fbc9f8285e6ffb3
Jonathan T. Barron
NeRF-Casting: Improved View-Dependent Appearance with Consistent
  Reflections
Neural Radiance Fields (NeRFs) typically struggle to reconstruct and render
highly specular objects, whose appearance varies quickly with changes in
viewpoint. Recent works have improved NeRF's ability to render detailed
specular appearance of distant environment illumination, but are unable to
synthesize consistent reflections of closer content. Moreover, these techniques
rely on large computationally-expensive neural networks to model outgoing
radiance, which severely limits optimization and rendering speed. We address
these issues with an approach based on ray tracing: instead of querying an
expensive neural network for the outgoing view-dependent radiance at points
along each camera ray, our model casts reflection rays from these points and
traces them through the NeRF representation to render feature vectors which are
decoded into color using a small inexpensive network. We demonstrate that our
model outperforms prior methods for view synthesis of scenes containing shiny
objects, and that it is the only existing NeRF method that can synthesize
photorealistic specular appearance and reflections in real-world scenes, while
requiring comparable optimization time to current state-of-the-art view
synthesis models.
The proposed method enhances NeRF's rendering of specular objects by using ray tracing and a small network to decode feature vectors, achieving photorealistic reflections with efficient optimization.
Neural Radiance Fields NeRFs specular objects ray tracing view-dependent radiance feature vectors view synthesis photorealistic specular appearance reflections