Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based
  View Synthesis
While surface-based view synthesis algorithms are appealing due to their low
computational requirements, they often struggle to reproduce thin structures.
In contrast, more expensive methods that model the scene's geometry as a
volumetric density field (e.g. NeRF) excel at reconstructing fine geometric
detail. However, density fields often represent geometry in a "fuzzy" manner,
which hinders exact localization of the surface. In this work, we modify
density fields to encourage them to converge towards surfaces, without
compromising their ability to reconstruct thin structures. First, we employ a
discrete opacity grid representation instead of a continuous density field,
which allows opacity values to discontinuously transition from zero to one at
the surface. Second, we anti-alias by casting multiple rays per pixel, which
allows occlusion boundaries and subpixel structures to be modelled without
using semi-transparent voxels. Third, we minimize the binary entropy of the
opacity values, which facilitates the extraction of surface geometry by
encouraging opacity values to binarize towards the end of training. Lastly, we
develop a fusion-based meshing strategy followed by mesh simplification and
appearance model fitting. The compact meshes produced by our model can be
rendered in real-time on mobile devices and achieve significantly higher view
synthesis quality compared to existing mesh-based approaches.
65d4436c9f087171b8fc4a3d
Christian Reiser
admin_assigned
2024-02-20T10:47:35.622Z
630b4b11a20a5367811975bd
/avatars/89b3351a6d542f82e7c8e8a7d352fe82.svg
Christian Reisert
user
HeavyRABBIT
65d4436c9f087171b8fc4a3e
Stephan Garbin
65d4436c9f087171b8fc4a3f
Pratul P. Srinivasan
65d4436c9f087171b8fc4a40
Dor Verbin
admin_assigned
2024-02-20T10:47:53.337Z
6315b0fc915d0b806830b424
/avatars/e25a7c8df2364c6e8024ec5c646a2889.svg
Dor Verbin
user
dorverbin
65d4436c9f087171b8fc4a41
Richard Szeliski
65d4436c9f087171b8fc4a42
Ben Mildenhall
65d4436c9f087171b8fc4a43
Jonathan T. Barron
65d4436c9f087171b8fc4a44
Peter Hedman
admin_assigned
2024-02-20T10:48:28.854Z
651ec3aee17ef6a94eefc8e7
/avatars/e90ce400ec0d2de8f9fe4365a6e07b4f.svg
Peter Hedman
user
phedman
65d4436c9f087171b8fc4a45
Andreas Geiger
admin_assigned
2024-02-20T10:48:35.650Z
620cae049086f3c07f01e3d5
https://cdn-avatars.huggingface.co/v1/production/uploads/1644998139538-noauth.jpeg
Andreas Geiger
user
andreas-geiger
Binary Opacity Grids: Capturing Fine Geometric Detail for Mesh-Based
  View Synthesis
While surface-based view synthesis algorithms are appealing due to their low
computational requirements, they often struggle to reproduce thin structures.
In contrast, more expensive methods that model the scene's geometry as a
volumetric density field (e.g. NeRF) excel at reconstructing fine geometric
detail. However, density fields often represent geometry in a "fuzzy" manner,
which hinders exact localization of the surface. In this work, we modify
density fields to encourage them to converge towards surfaces, without
compromising their ability to reconstruct thin structures. First, we employ a
discrete opacity grid representation instead of a continuous density field,
which allows opacity values to discontinuously transition from zero to one at
the surface. Second, we anti-alias by casting multiple rays per pixel, which
allows occlusion boundaries and subpixel structures to be modelled without
using semi-transparent voxels. Third, we minimize the binary entropy of the
opacity values, which facilitates the extraction of surface geometry by
encouraging opacity values to binarize towards the end of training. Lastly, we
develop a fusion-based meshing strategy followed by mesh simplification and
appearance model fitting. The compact meshes produced by our model can be
rendered in real-time on mobile devices and achieve significantly higher view
synthesis quality compared to existing mesh-based approaches.
The work modifies volumetric density fields to better represent surfaces and thin structures, enabling higher-quality view synthesis with real-time rendering on mobile devices.
volumetric density field NeRF discrete opacity grid anti-aliasing binary entropy meshing strategy mesh simplification appearance model fitting