NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to
  Robotics
This work was presented at the IEEE International Conference on Robotics and
Automation 2023 Workshop on Unconventional Spatial Representations.
  Neural radiance fields (NeRFs) are a class of implicit scene representations
that model 3D environments from color images. NeRFs are expressive, and can
model the complex and multi-scale geometry of real world environments, which
potentially makes them a powerful tool for robotics applications. Modern NeRF
training libraries can generate a photo-realistic NeRF from a static data set
in just a few seconds, but are designed for offline use and require a slow pose
optimization pre-computation step.
  In this work we propose NerfBridge, an open-source bridge between the Robot
Operating System (ROS) and the popular Nerfstudio library for real-time, online
training of NeRFs from a stream of images. NerfBridge enables rapid development
of research on applications of NeRFs in robotics by providing an extensible
interface to the efficient training pipelines and model libraries provided by
Nerfstudio. As an example use case we outline a hardware setup that can be used
NerfBridge to train a NeRF from images captured by a camera mounted to a
quadrotor in both indoor and outdoor environments.
  For accompanying video https://youtu.be/EH0SLn-RcDg and code
https://github.com/javieryu/nerf_bridge.
646596410e6c7618f6157389
Javier Yu
admin_assigned
2023-05-21T20:46:50.752Z
6468d81f188ca2e6bc58dcf8
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/aWR-pxpHVa4aJ-5ajk_Iy.jpeg
Javier Yu
user
javieryu
646596410e6c7618f615738a
Jun En Low
646596410e6c7618f615738b
Keiko Nagami
646596410e6c7618f615738c
Mac Schwager
NerfBridge: Bringing Real-time, Online Neural Radiance Field Training to
  Robotics
This work was presented at the IEEE International Conference on Robotics and
Automation 2023 Workshop on Unconventional Spatial Representations.
  Neural radiance fields (NeRFs) are a class of implicit scene representations
that model 3D environments from color images. NeRFs are expressive, and can
model the complex and multi-scale geometry of real world environments, which
potentially makes them a powerful tool for robotics applications. Modern NeRF
training libraries can generate a photo-realistic NeRF from a static data set
in just a few seconds, but are designed for offline use and require a slow pose
optimization pre-computation step.
  In this work we propose NerfBridge, an open-source bridge between the Robot
Operating System (ROS) and the popular Nerfstudio library for real-time, online
training of NeRFs from a stream of images. NerfBridge enables rapid development
of research on applications of NeRFs in robotics by providing an extensible
interface to the efficient training pipelines and model libraries provided by
Nerfstudio. As an example use case we outline a hardware setup that can be used
NerfBridge to train a NeRF from images captured by a camera mounted to a
quadrotor in both indoor and outdoor environments.
  For accompanying video https://youtu.be/EH0SLn-RcDg and code
https://github.com/javieryu/nerf_bridge.
NerfBridge facilitates real-time NeRF training in robotics by integrating ROS with Nerfstudio, enabling rapid development for various applications.
Neural radiance fields NeRFs implicit scene representations Robot Operating System ROS Nerfstudio real-time training online training quadrotor indoor environments outdoor environments