LightSpeed: Light and Fast Neural Light Fields on Mobile Devices
Real-time novel-view image synthesis on mobile devices is prohibitive due to
the limited computational power and storage. Using volumetric rendering
methods, such as NeRF and its derivatives, on mobile devices is not suitable
due to the high computational cost of volumetric rendering. On the other hand,
recent advances in neural light field representations have shown promising
real-time view synthesis results on mobile devices. Neural light field methods
learn a direct mapping from a ray representation to the pixel color. The
current choice of ray representation is either stratified ray sampling or
Pl\"{u}cker coordinates, overlooking the classic light slab (two-plane)
representation, the preferred representation to interpolate between light field
views. In this work, we find that using the light slab representation is an
efficient representation for learning a neural light field. More importantly,
it is a lower-dimensional ray representation enabling us to learn the 4D ray
space using feature grids which are significantly faster to train and render.
Although mostly designed for frontal views, we show that the light-slab
representation can be further extended to non-frontal scenes using a
divide-and-conquer strategy. Our method offers superior rendering quality
compared to previous light field methods and achieves a significantly improved
trade-off between rendering quality and speed.
6539dd390d111541df1f09ad
Aarush Gupta
claimed_verified
2023-10-26T18:19:16.795Z
653a7dc5d9698d72fd341ae3
/avatars/51a1f63f25e6d069eb133ceb3b62269e.svg
Aarush Gupta
user
aarush98
6539dd390d111541df1f09ae
Junli Cao
admin_assigned
2023-10-26T08:29:29.201Z
63f54aa73aa49d8cb97b84bc
/avatars/c73c5870039611ab9162daad46a1ba20.svg
junli cao
user
jlcao2
6539dd390d111541df1f09af
Chaoyang Wang
admin_assigned
2023-10-26T08:29:36.523Z
64c166de11f3f1d23a99c6fd
/avatars/d4730f70a6f1cb51266862c7a8d54f77.svg
Chaoyang Wang
user
cwang9
6539dd390d111541df1f09b0
Ju Hu
6539dd390d111541df1f09b1
Sergey Tulyakov
6539dd390d111541df1f09b2
Jian Ren
claimed_verified
2023-10-26T07:32:31.108Z
61f19829233c91cbd2f79e70
/avatars/a0735a94542b4f7cda5aed8bc4be0538.svg
Jian Ren
user
alanspike
6539dd390d111541df1f09b3
László A Jeni
LightSpeed: Light and Fast Neural Light Fields on Mobile Devices
Real-time novel-view image synthesis on mobile devices is prohibitive due to
the limited computational power and storage. Using volumetric rendering
methods, such as NeRF and its derivatives, on mobile devices is not suitable
due to the high computational cost of volumetric rendering. On the other hand,
recent advances in neural light field representations have shown promising
real-time view synthesis results on mobile devices. Neural light field methods
learn a direct mapping from a ray representation to the pixel color. The
current choice of ray representation is either stratified ray sampling or
Pl\"{u}cker coordinates, overlooking the classic light slab (two-plane)
representation, the preferred representation to interpolate between light field
views. In this work, we find that using the light slab representation is an
efficient representation for learning a neural light field. More importantly,
it is a lower-dimensional ray representation enabling us to learn the 4D ray
space using feature grids which are significantly faster to train and render.
Although mostly designed for frontal views, we show that the light-slab
representation can be further extended to non-frontal scenes using a
divide-and-conquer strategy. Our method offers superior rendering quality
compared to previous light field methods and achieves a significantly improved
trade-off between rendering quality and speed.
Neural light field methods using light slab representation offer superior rendering quality and speed on mobile devices compared to previous approaches.
NeRF volumetric rendering neural light field ray representation stratified ray sampling Plücker coordinates light slab 4D ray space feature grids divide-and-conquer strategy