MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and
  Photorealism From Sparse Views
We present a novel appearance model that simultaneously realizes explicit
high-quality 3D surface mesh recovery and photorealistic novel view synthesis
from sparse view samples. Our key idea is to model the underlying scene
geometry Mesh as an Atlas of Charts which we render with 2D Gaussian surfels
(MAtCha Gaussians). MAtCha distills high-frequency scene surface details from
an off-the-shelf monocular depth estimator and refines it through Gaussian
surfel rendering. The Gaussian surfels are attached to the charts on the fly,
satisfying photorealism of neural volumetric rendering and crisp geometry of a
mesh model, i.e., two seemingly contradicting goals in a single model. At the
core of MAtCha lies a novel neural deformation model and a structure loss that
preserve the fine surface details distilled from learned monocular depths while
addressing their fundamental scale ambiguities. Results of extensive
experimental validation demonstrate MAtCha's state-of-the-art quality of
surface reconstruction and photorealism on-par with top contenders but with
dramatic reduction in the number of input views and computational time. We
believe MAtCha will serve as a foundational tool for any visual application in
vision, graphics, and robotics that require explicit geometry in addition to
photorealism. Our project page is the following:
https://anttwo.github.io/matcha/
6758776c9dc2cbfe5f45ee7f
Antoine Guédon
extracted_confirmed
2024-12-10T18:24:51.244Z
6489dd894bbed6144c870274
https://cdn-avatars.huggingface.co/v1/production/uploads/6489dd894bbed6144c870274/c_J9b4DHwXeE5VtTeQ2ue.jpeg
Antoine Guédon
user
AntoineGuedon
6758776c9dc2cbfe5f45ee80
Tomoki Ichikawa
6758776c9dc2cbfe5f45ee81
Kohei Yamashita
6758776c9dc2cbfe5f45ee82
Ko Nishino
MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and
  Photorealism From Sparse Views
We present a novel appearance model that simultaneously realizes explicit
high-quality 3D surface mesh recovery and photorealistic novel view synthesis
from sparse view samples. Our key idea is to model the underlying scene
geometry Mesh as an Atlas of Charts which we render with 2D Gaussian surfels
(MAtCha Gaussians). MAtCha distills high-frequency scene surface details from
an off-the-shelf monocular depth estimator and refines it through Gaussian
surfel rendering. The Gaussian surfels are attached to the charts on the fly,
satisfying photorealism of neural volumetric rendering and crisp geometry of a
mesh model, i.e., two seemingly contradicting goals in a single model. At the
core of MAtCha lies a novel neural deformation model and a structure loss that
preserve the fine surface details distilled from learned monocular depths while
addressing their fundamental scale ambiguities. Results of extensive
experimental validation demonstrate MAtCha's state-of-the-art quality of
surface reconstruction and photorealism on-par with top contenders but with
dramatic reduction in the number of input views and computational time. We
believe MAtCha will serve as a foundational tool for any visual application in
vision, graphics, and robotics that require explicit geometry in addition to
photorealism. Our project page is the following:
https://anttwo.github.io/matcha/
MAtCha, a novel appearance model, combines 3D surface mesh recovery and photorealistic novel view synthesis using an atlas of charts rendered with Gaussian surfels, achieving high-quality results with fewer input views and less computation.
MAtCha Gaussians Gaussian surfels atlas of charts neural volumetric rendering neural deformation model structure loss monocular depth estimator surface reconstruction