Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
Neural rendering methods have significantly advanced photo-realistic 3D scene
rendering in various academic and industrial applications. The recent 3D
Gaussian Splatting method has achieved the state-of-the-art rendering quality
and speed combining the benefits of both primitive-based representations and
volumetric representations. However, it often leads to heavily redundant
Gaussians that try to fit every training view, neglecting the underlying scene
geometry. Consequently, the resulting model becomes less robust to significant
view changes, texture-less area and lighting effects. We introduce Scaffold-GS,
which uses anchor points to distribute local 3D Gaussians, and predicts their
attributes on-the-fly based on viewing direction and distance within the view
frustum. Anchor growing and pruning strategies are developed based on the
importance of neural Gaussians to reliably improve the scene coverage. We show
that our method effectively reduces redundant Gaussians while delivering
high-quality rendering. We also demonstrates an enhanced capability to
accommodate scenes with varying levels-of-detail and view-dependent
observations, without sacrificing the rendering speed.
656d40a08a37acfa3f1f051a
Tao Lu
claimed_verified
2023-12-05T08:40:36.885Z
656e9db559afca5707c493c5
/avatars/ffe72a82aaf42c80eb988344415672ba.svg
Tao Lu
user
Isaaclt
656d40a08a37acfa3f1f051b
Mulin Yu
claimed_verified
2023-12-05T08:40:33.708Z
656e9e26bbec423d88b603e8
/avatars/10d8cb945a60e0401bfa4f74137cb203.svg
MulinYu
user
UML
656d40a08a37acfa3f1f051c
Linning Xu
656d40a08a37acfa3f1f051d
Yuanbo Xiangli
admin_assigned
2023-12-04T11:04:13.845Z
651edd0faa888068acb5d680
/avatars/d5e03e8bc1d22bc67a3e684b52dc5598.svg
Yuanbo Xiangli
user
amberkk
656d40a08a37acfa3f1f051e
Limin Wang
admin_assigned
2023-12-04T11:05:17.404Z
62c77f4352d8ae531f5511f9
/avatars/50198ccb02ccd286975a4613fbabee28.svg
Limin Wang
user
lmwang
656d40a08a37acfa3f1f051f
Dahua Lin
admin_assigned
2023-12-04T11:05:26.007Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
656d40a08a37acfa3f1f0520
Bo Dai
admin_assigned
2023-12-04T11:05:34.052Z
635f93577c05eb9f59966209
/avatars/add48d7e3790a6b500d6c451ef8b0f75.svg
Intelligent Digital Creation
user
BoDai
Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering
Neural rendering methods have significantly advanced photo-realistic 3D scene
rendering in various academic and industrial applications. The recent 3D
Gaussian Splatting method has achieved the state-of-the-art rendering quality
and speed combining the benefits of both primitive-based representations and
volumetric representations. However, it often leads to heavily redundant
Gaussians that try to fit every training view, neglecting the underlying scene
geometry. Consequently, the resulting model becomes less robust to significant
view changes, texture-less area and lighting effects. We introduce Scaffold-GS,
which uses anchor points to distribute local 3D Gaussians, and predicts their
attributes on-the-fly based on viewing direction and distance within the view
frustum. Anchor growing and pruning strategies are developed based on the
importance of neural Gaussians to reliably improve the scene coverage. We show
that our method effectively reduces redundant Gaussians while delivering
high-quality rendering. We also demonstrates an enhanced capability to
accommodate scenes with varying levels-of-detail and view-dependent
observations, without sacrificing the rendering speed.
Scaffold-GS improves 3D scene rendering by reducing Gaussian redundancy and enhancing robustness to varying views and lighting using anchor points and neural Gaussian importance assessment.
Neural rendering 3D Gaussian Splatting primitive-based representations volumetric representations Gaussians anchor points attribute prediction view frustum anchor growing anchor pruning scene geometry rendering quality view-dependent observations levels-of-detail