Neural Directional Encoding for Efficient and Accurate View-Dependent
  Appearance Modeling
Novel-view synthesis of specular objects like shiny metals or glossy paints
remains a significant challenge. Not only the glossy appearance but also global
illumination effects, including reflections of other objects in the
environment, are critical components to faithfully reproduce a scene. In this
paper, we present Neural Directional Encoding (NDE), a view-dependent
appearance encoding of neural radiance fields (NeRF) for rendering specular
objects. NDE transfers the concept of feature-grid-based spatial encoding to
the angular domain, significantly improving the ability to model high-frequency
angular signals. In contrast to previous methods that use encoding functions
with only angular input, we additionally cone-trace spatial features to obtain
a spatially varying directional encoding, which addresses the challenging
interreflection effects. Extensive experiments on both synthetic and real
datasets show that a NeRF model with NDE (1) outperforms the state of the art
on view synthesis of specular objects, and (2) works with small networks to
allow fast (real-time) inference. The project webpage and source code are
available at: https://lwwu2.github.io/nde/.
66509bf987f3322826fb775f
Liwen Wu
admin_assigned
2024-05-24T15:17:51.657Z
662b680ef6a60e8e58b15673
/avatars/b1f4636511903092cf6bf286cedb600c.svg
liwenwu
user
liwenwu
66509bf987f3322826fb7760
Sai Bi
admin_assigned
2024-05-24T15:17:58.101Z
66303560597dc7c535168f07
/avatars/a69d52a4a198c6c8451c1d137031bf07.svg
Sai Bi
user
saibi
66509bf987f3322826fb7761
Zexiang Xu
admin_assigned
2024-05-24T15:18:05.983Z
65565b1b8616d8d740abb976
/avatars/c245b19ed05292bf1bf4a9d8cf2a409c.svg
Zexiang Xu
user
zexiangxu
66509bf987f3322826fb7762
Fujun Luan
admin_assigned
2024-05-24T15:18:12.137Z
662ae0aa4f711ee4e1e17ffc
https://cdn-avatars.huggingface.co/v1/production/uploads/662ae0aa4f711ee4e1e17ffc/nEi6tv8f6DIhTauwHpw_C.jpeg
Fujun Luan
user
luanfujun
66509bf987f3322826fb7763
Kai Zhang
66509bf987f3322826fb7764
Iliyan Georgiev
admin_assigned
2024-05-24T15:18:47.186Z
651aea3ec6d84a147b1c6ade
/avatars/c387b90d618d195d0338b89b2d0bcfea.svg
Iliyan Georgiev
user
iliyan-g
66509bf987f3322826fb7765
Kalyan Sunkavalli
66509bf987f3322826fb7766
Ravi Ramamoorthi
Neural Directional Encoding for Efficient and Accurate View-Dependent
  Appearance Modeling
Novel-view synthesis of specular objects like shiny metals or glossy paints
remains a significant challenge. Not only the glossy appearance but also global
illumination effects, including reflections of other objects in the
environment, are critical components to faithfully reproduce a scene. In this
paper, we present Neural Directional Encoding (NDE), a view-dependent
appearance encoding of neural radiance fields (NeRF) for rendering specular
objects. NDE transfers the concept of feature-grid-based spatial encoding to
the angular domain, significantly improving the ability to model high-frequency
angular signals. In contrast to previous methods that use encoding functions
with only angular input, we additionally cone-trace spatial features to obtain
a spatially varying directional encoding, which addresses the challenging
interreflection effects. Extensive experiments on both synthetic and real
datasets show that a NeRF model with NDE (1) outperforms the state of the art
on view synthesis of specular objects, and (2) works with small networks to
allow fast (real-time) inference. The project webpage and source code are
available at: https://lwwu2.github.io/nde/.
Neural Directional Encoding enhances neural radiance fields for rendering specular objects by incorporating angular encoding and spatially varying directional encoding to model reflections and improve real-time performance.
Neural Directional Encoding neural radiance fields NDE feature-grid-based spatial encoding angular domain high-frequency angular signals cone-trace spatial features interreflection effects view synthesis specular objects real-time inference