ThermalNeRF: Thermal Radiance Fields
Thermal imaging has a variety of applications, from agricultural monitoring
to building inspection to imaging under poor visibility, such as in low light,
fog, and rain. However, reconstructing thermal scenes in 3D presents several
challenges due to the comparatively lower resolution and limited features
present in long-wave infrared (LWIR) images. To overcome these challenges, we
propose a unified framework for scene reconstruction from a set of LWIR and RGB
images, using a multispectral radiance field to represent a scene viewed by
both visible and infrared cameras, thus leveraging information across both
spectra. We calibrate the RGB and infrared cameras with respect to each other,
as a preprocessing step using a simple calibration target. We demonstrate our
method on real-world sets of RGB and LWIR photographs captured from a handheld
thermal camera, showing the effectiveness of our method at scene representation
across the visible and infrared spectra. We show that our method is capable of
thermal super-resolution, as well as visually removing obstacles to reveal
objects that are occluded in either the RGB or thermal channels. Please see
https://yvette256.github.io/thermalnerf for video results as well as our code
and dataset release.
669f3dcaece12f050c2574ba
Yvette Y. Lin
669f3dcaece12f050c2574bb
Xin-Yi Pan
669f3dcaece12f050c2574bc
Sara Fridovich-Keil
669f3dcaece12f050c2574bd
Gordon Wetzstein
admin_assigned
2024-07-23T12:52:09.832Z
6694e583ac96ca2c17131505
/avatars/6e7a31f257e36cf301da6f879dc0a122.svg
Gordon Wetzstein
user
wetzste1
ThermalNeRF: Thermal Radiance Fields
Thermal imaging has a variety of applications, from agricultural monitoring
to building inspection to imaging under poor visibility, such as in low light,
fog, and rain. However, reconstructing thermal scenes in 3D presents several
challenges due to the comparatively lower resolution and limited features
present in long-wave infrared (LWIR) images. To overcome these challenges, we
propose a unified framework for scene reconstruction from a set of LWIR and RGB
images, using a multispectral radiance field to represent a scene viewed by
both visible and infrared cameras, thus leveraging information across both
spectra. We calibrate the RGB and infrared cameras with respect to each other,
as a preprocessing step using a simple calibration target. We demonstrate our
method on real-world sets of RGB and LWIR photographs captured from a handheld
thermal camera, showing the effectiveness of our method at scene representation
across the visible and infrared spectra. We show that our method is capable of
thermal super-resolution, as well as visually removing obstacles to reveal
objects that are occluded in either the RGB or thermal channels. Please see
https://yvette256.github.io/thermalnerf for video results as well as our code
and dataset release.
A framework using multispectral radiance fields and calibrate cameras for effective 3D reconstruction and thermal super-resolution from RGB and LWIR images.
multispectral radiance field