Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field
  Representation and Generation
We present Mesh2NeRF, an approach to derive ground-truth radiance fields from
textured meshes for 3D generation tasks. Many 3D generative approaches
represent 3D scenes as radiance fields for training. Their ground-truth
radiance fields are usually fitted from multi-view renderings from a
large-scale synthetic 3D dataset, which often results in artifacts due to
occlusions or under-fitting issues. In Mesh2NeRF, we propose an analytic
solution to directly obtain ground-truth radiance fields from 3D meshes,
characterizing the density field with an occupancy function featuring a defined
surface thickness, and determining view-dependent color through a reflection
function considering both the mesh and environment lighting. Mesh2NeRF extracts
accurate radiance fields which provides direct supervision for training
generative NeRFs and single scene representation. We validate the effectiveness
of Mesh2NeRF across various tasks, achieving a noteworthy 3.12dB improvement in
PSNR for view synthesis in single scene representation on the ABO dataset, a
0.69 PSNR enhancement in the single-view conditional generation of ShapeNet
Cars, and notably improved mesh extraction from NeRF in the unconditional
generation of Objaverse Mugs.
66063cf957778f7a192ec4bf
Yujin Chen
claimed_verified
2024-09-23T13:07:18.787Z
63c307926e6561b339ddebb2
/avatars/32548c8495494ab7682d5983fbca315e.svg
Yujin Chen
user
YujinChen
66063cf957778f7a192ec4c0
Yinyu Nie
66063cf957778f7a192ec4c1
Benjamin Ummenhofer
66063cf957778f7a192ec4c2
Reiner Birkl
66063cf957778f7a192ec4c3
Michael Paulitsch
66063cf957778f7a192ec4c4
Matthias Müller
66063cf957778f7a192ec4c5
Matthias Nießner
Mesh2NeRF: Direct Mesh Supervision for Neural Radiance Field
  Representation and Generation
We present Mesh2NeRF, an approach to derive ground-truth radiance fields from
textured meshes for 3D generation tasks. Many 3D generative approaches
represent 3D scenes as radiance fields for training. Their ground-truth
radiance fields are usually fitted from multi-view renderings from a
large-scale synthetic 3D dataset, which often results in artifacts due to
occlusions or under-fitting issues. In Mesh2NeRF, we propose an analytic
solution to directly obtain ground-truth radiance fields from 3D meshes,
characterizing the density field with an occupancy function featuring a defined
surface thickness, and determining view-dependent color through a reflection
function considering both the mesh and environment lighting. Mesh2NeRF extracts
accurate radiance fields which provides direct supervision for training
generative NeRFs and single scene representation. We validate the effectiveness
of Mesh2NeRF across various tasks, achieving a noteworthy 3.12dB improvement in
PSNR for view synthesis in single scene representation on the ABO dataset, a
0.69 PSNR enhancement in the single-view conditional generation of ShapeNet
Cars, and notably improved mesh extraction from NeRF in the unconditional
generation of Objaverse Mugs.
Mesh2NeRF derives ground-truth radiance fields from 3D meshes using an occupancy function and reflection function, improving accuracy for NeRF training and generation.
radiance fields textured meshes 3D generation occupancy function reflection function NeRF ABO dataset ShapeNet Cars Objaverse Mugs PSNR view synthesis single-view conditional generation unconditional generation