FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally
This study addresses the challenge of accurately segmenting 3D Gaussian
Splatting from 2D masks. Conventional methods often rely on iterative gradient
descent to assign each Gaussian a unique label, leading to lengthy optimization
and sub-optimal solutions. Instead, we propose a straightforward yet globally
optimal solver for 3D-GS segmentation. The core insight of our method is that,
with a reconstructed 3D-GS scene, the rendering of the 2D masks is essentially
a linear function with respect to the labels of each Gaussian. As such, the
optimal label assignment can be solved via linear programming in closed form.
This solution capitalizes on the alpha blending characteristic of the splatting
process for single step optimization. By incorporating the background bias in
our objective function, our method shows superior robustness in 3D segmentation
against noises. Remarkably, our optimization completes within 30 seconds, about
50times faster than the best existing methods. Extensive experiments
demonstrate the efficiency and robustness of our method in segmenting various
scenes, and its superior performance in downstream tasks such as object removal
and inpainting. Demos and code will be available at
https://github.com/florinshen/FlashSplat.
66e39c30f5579b829fe5e4f1
Qiuhong Shen
admin_assigned
2024-09-13T08:05:33.263Z
643a6e89a856622f9788bf67
/avatars/419c0379f072295b27d4bfe2f8fb946d.svg
qiuhong shen
user
florinshum
66e39c30f5579b829fe5e4f2
Xingyi Yang
admin_assigned
2024-09-13T08:05:22.780Z
634cfebc350bcee9bed20a4d
https://cdn-avatars.huggingface.co/v1/production/uploads/634cfebc350bcee9bed20a4d/fN47nN5rhw-HJaFLBZWQy.png
Xingyi Yang
user
adamdad
66e39c30f5579b829fe5e4f3
Xinchao Wang
admin_assigned
2024-09-13T08:06:21.872Z
63fc03a50aab060792ffef39
/avatars/9d5b1bb2a41928e08176b703935133ab.svg
Wangxinchao
user
wxcTest
FlashSplat: 2D to 3D Gaussian Splatting Segmentation Solved Optimally
This study addresses the challenge of accurately segmenting 3D Gaussian
Splatting from 2D masks. Conventional methods often rely on iterative gradient
descent to assign each Gaussian a unique label, leading to lengthy optimization
and sub-optimal solutions. Instead, we propose a straightforward yet globally
optimal solver for 3D-GS segmentation. The core insight of our method is that,
with a reconstructed 3D-GS scene, the rendering of the 2D masks is essentially
a linear function with respect to the labels of each Gaussian. As such, the
optimal label assignment can be solved via linear programming in closed form.
This solution capitalizes on the alpha blending characteristic of the splatting
process for single step optimization. By incorporating the background bias in
our objective function, our method shows superior robustness in 3D segmentation
against noises. Remarkably, our optimization completes within 30 seconds, about
50times faster than the best existing methods. Extensive experiments
demonstrate the efficiency and robustness of our method in segmenting various
scenes, and its superior performance in downstream tasks such as object removal
and inpainting. Demos and code will be available at
https://github.com/florinshen/FlashSplat.
A globally optimal linear programming solver for 3D Gaussian Splatting segmentation is proposed, offering significant speed and robustness improvements.
Gaussian Splatting linear programming alpha blending background bias