ResFields: Residual Neural Fields for Spatiotemporal Signals
Neural fields, a category of neural networks trained to represent
high-frequency signals, have gained significant attention in recent years due
to their impressive performance in modeling complex 3D data, especially large
neural signed distance (SDFs) or radiance fields (NeRFs) via a single
multi-layer perceptron (MLP). However, despite the power and simplicity of
representing signals with an MLP, these methods still face challenges when
modeling large and complex temporal signals due to the limited capacity of
MLPs. In this paper, we propose an effective approach to address this
limitation by incorporating temporal residual layers into neural fields, dubbed
ResFields, a novel class of networks specifically designed to effectively
represent complex temporal signals. We conduct a comprehensive analysis of the
properties of ResFields and propose a matrix factorization technique to reduce
the number of trainable parameters and enhance generalization capabilities.
Importantly, our formulation seamlessly integrates with existing techniques and
consistently improves results across various challenging tasks: 2D video
approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF
reconstruction. Lastly, we demonstrate the practical utility of ResFields by
showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory
inputs of a lightweight capture system.
64f949728a234f114e4731ec
Marko Mihajlovic
admin_assigned
2023-09-12T13:31:01.950Z
62f7eddeb9fda55613cb9f03
https://cdn-avatars.huggingface.co/v1/production/uploads/1660415445288-noauth.jpeg
Marko Mihajlovic
user
markomih
64f949728a234f114e4731ed
Sergey Prokudin
admin_assigned
2023-09-12T13:31:15.323Z
6340364b733f9eef469165ef
/avatars/c86d79f1588bb625b97dcbe471e46def.svg
Sergey Prokudin
user
sprokudin
64f949728a234f114e4731ee
Marc Pollefeys
claimed_verified
2025-02-25T09:43:01.582Z
67b5fa179782a5e2fd2cb26a
/avatars/62c38f29ec641e001eeddf840bea21a0.svg
Marc Pollefeys
user
mapo1
64f949728a234f114e4731ef
Siyu Tang
ResFields: Residual Neural Fields for Spatiotemporal Signals
Neural fields, a category of neural networks trained to represent
high-frequency signals, have gained significant attention in recent years due
to their impressive performance in modeling complex 3D data, especially large
neural signed distance (SDFs) or radiance fields (NeRFs) via a single
multi-layer perceptron (MLP). However, despite the power and simplicity of
representing signals with an MLP, these methods still face challenges when
modeling large and complex temporal signals due to the limited capacity of
MLPs. In this paper, we propose an effective approach to address this
limitation by incorporating temporal residual layers into neural fields, dubbed
ResFields, a novel class of networks specifically designed to effectively
represent complex temporal signals. We conduct a comprehensive analysis of the
properties of ResFields and propose a matrix factorization technique to reduce
the number of trainable parameters and enhance generalization capabilities.
Importantly, our formulation seamlessly integrates with existing techniques and
consistently improves results across various challenging tasks: 2D video
approximation, dynamic shape modeling via temporal SDFs, and dynamic NeRF
reconstruction. Lastly, we demonstrate the practical utility of ResFields by
showcasing its effectiveness in capturing dynamic 3D scenes from sparse sensory
inputs of a lightweight capture system.
ResFields, a neural network variant with temporal residual layers, enhance the representation of complex temporal signals in neural fields, improving performance across 2D video approximation, dynamic shape modeling, and NeRF reconstruction.
neural fields neural signed distance radiance fields multi-layer perceptron MLP temporal residual layers ResFields matrix factorization generalization capabilities 2D video approximation dynamic shape modeling temporal SDFs dynamic NeRF reconstruction 3D scenes sparse sensory inputs