Towards Realistic Example-based Modeling via 3D Gaussian Stitching
Using parts of existing models to rebuild new models, commonly termed as
example-based modeling, is a classical methodology in the realm of computer
graphics. Previous works mostly focus on shape composition, making them very
hard to use for realistic composition of 3D objects captured from real-world
scenes. This leads to combining multiple NeRFs into a single 3D scene to
achieve seamless appearance blending. However, the current SeamlessNeRF method
struggles to achieve interactive editing and harmonious stitching for
real-world scenes due to its gradient-based strategy and grid-based
representation. To this end, we present an example-based modeling method that
combines multiple Gaussian fields in a point-based representation using
sample-guided synthesis. Specifically, as for composition, we create a GUI to
segment and transform multiple fields in real time, easily obtaining a
semantically meaningful composition of models represented by 3D Gaussian
Splatting (3DGS). For texture blending, due to the discrete and irregular
nature of 3DGS, straightforwardly applying gradient propagation as SeamlssNeRF
is not supported. Thus, a novel sampling-based cloning method is proposed to
harmonize the blending while preserving the original rich texture and content.
Our workflow consists of three steps: 1) real-time segmentation and
transformation of a Gaussian model using a well-tailored GUI, 2) KNN analysis
to identify boundary points in the intersecting area between the source and
target models, and 3) two-phase optimization of the target model using
sampling-based cloning and gradient constraints. Extensive experimental results
validate that our approach significantly outperforms previous works in terms of
realistic synthesis, demonstrating its practicality. More demos are available
at https://ingra14m.github.io/gs_stitching_website.
66cfdb63be74f57bd49d39bd
Xinyu Gao
66cfdb63be74f57bd49d39be
Ziyi Yang
extracted_pending
2024-08-29T02:22:32.743Z
65d6b7f9e77bb42e9a652c7f
/avatars/defa84e5df00a6599c217daffbcd4e08.svg
Zyeir
user
Ingra14m
66cfdb63be74f57bd49d39bf
Bingchen Gong
extracted_pending
2024-08-29T02:22:32.743Z
634938bcaa45636285efda4c
/avatars/eb2f8d45be6b8396b446a0bbeaba7870.svg
Wenri
user
wenri
66cfdb63be74f57bd49d39c0
Xiaoguang Han
66cfdb63be74f57bd49d39c1
Sipeng Yang
66cfdb63be74f57bd49d39c2
Xiaogang Jin
Towards Realistic Example-based Modeling via 3D Gaussian Stitching
Using parts of existing models to rebuild new models, commonly termed as
example-based modeling, is a classical methodology in the realm of computer
graphics. Previous works mostly focus on shape composition, making them very
hard to use for realistic composition of 3D objects captured from real-world
scenes. This leads to combining multiple NeRFs into a single 3D scene to
achieve seamless appearance blending. However, the current SeamlessNeRF method
struggles to achieve interactive editing and harmonious stitching for
real-world scenes due to its gradient-based strategy and grid-based
representation. To this end, we present an example-based modeling method that
combines multiple Gaussian fields in a point-based representation using
sample-guided synthesis. Specifically, as for composition, we create a GUI to
segment and transform multiple fields in real time, easily obtaining a
semantically meaningful composition of models represented by 3D Gaussian
Splatting (3DGS). For texture blending, due to the discrete and irregular
nature of 3DGS, straightforwardly applying gradient propagation as SeamlssNeRF
is not supported. Thus, a novel sampling-based cloning method is proposed to
harmonize the blending while preserving the original rich texture and content.
Our workflow consists of three steps: 1) real-time segmentation and
transformation of a Gaussian model using a well-tailored GUI, 2) KNN analysis
to identify boundary points in the intersecting area between the source and
target models, and 3) two-phase optimization of the target model using
sampling-based cloning and gradient constraints. Extensive experimental results
validate that our approach significantly outperforms previous works in terms of
realistic synthesis, demonstrating its practicality. More demos are available
at https://ingra14m.github.io/gs_stitching_website.
A novel example-based modeling method combines multiple Gaussian fields using a point-based representation for realistic 3D object composition and texture blending, outperforming previous approaches.
Gaussian fields point-based representation sample-guided synthesis 3D Gaussian Splatting (3DGS) KNN analysis sampling-based cloning gradient constraints