GaussianSR: 3D Gaussian Super-Resolution with 2D Diffusion Priors
Achieving high-resolution novel view synthesis (HRNVS) from low-resolution
input views is a challenging task due to the lack of high-resolution data.
Previous methods optimize high-resolution Neural Radiance Field (NeRF) from
low-resolution input views but suffer from slow rendering speed. In this work,
we base our method on 3D Gaussian Splatting (3DGS) due to its capability of
producing high-quality images at a faster rendering speed. To alleviate the
shortage of data for higher-resolution synthesis, we propose to leverage
off-the-shelf 2D diffusion priors by distilling the 2D knowledge into 3D with
Score Distillation Sampling (SDS). Nevertheless, applying SDS directly to
Gaussian-based 3D super-resolution leads to undesirable and redundant 3D
Gaussian primitives, due to the randomness brought by generative priors. To
mitigate this issue, we introduce two simple yet effective techniques to reduce
stochastic disturbances introduced by SDS. Specifically, we 1) shrink the range
of diffusion timestep in SDS with an annealing strategy; 2) randomly discard
redundant Gaussian primitives during densification. Extensive experiments have
demonstrated that our proposed GaussainSR can attain high-quality results for
HRNVS with only low-resolution inputs on both synthetic and real-world
datasets. Project page: https://chchnii.github.io/GaussianSR/
666fe9a81f975c1f86db1d51
Xiqian Yu
extracted_confirmed
2024-06-17T08:36:47.720Z
619b7b1cab4c7b7f16a7d59e
https://cdn-avatars.huggingface.co/v1/production/uploads/619b7b1cab4c7b7f16a7d59e/6TvXaAqBghAMYO1-j5l4v.jpeg
Tianyu He
user
deeptimhe
666fe9a81f975c1f86db1d52
Hanxin Zhu
admin_assigned
2024-06-17T16:20:13.935Z
650e5beff874d950df2203f0
/avatars/384b35ea59b55f8d454be9efb7cce47c.svg
Hanxin ZHU
user
ingzzzz
666fe9a81f975c1f86db1d53
Tianyu He
admin_assigned
2024-06-17T16:20:47.708Z
619b7b1cab4c7b7f16a7d59e
https://cdn-avatars.huggingface.co/v1/production/uploads/619b7b1cab4c7b7f16a7d59e/6TvXaAqBghAMYO1-j5l4v.jpeg
Tianyu He
user
deeptimhe
666fe9a81f975c1f86db1d54
Zhibo Chen
claimed_verified
2024-06-18T07:18:59.227Z
666fe497fb4f22815598fb77
/avatars/dc61b962c11b916b750b017fe8010493.svg
Xiqian Yu
user
chchnii
GaussianSR: 3D Gaussian Super-Resolution with 2D Diffusion Priors
Achieving high-resolution novel view synthesis (HRNVS) from low-resolution
input views is a challenging task due to the lack of high-resolution data.
Previous methods optimize high-resolution Neural Radiance Field (NeRF) from
low-resolution input views but suffer from slow rendering speed. In this work,
we base our method on 3D Gaussian Splatting (3DGS) due to its capability of
producing high-quality images at a faster rendering speed. To alleviate the
shortage of data for higher-resolution synthesis, we propose to leverage
off-the-shelf 2D diffusion priors by distilling the 2D knowledge into 3D with
Score Distillation Sampling (SDS). Nevertheless, applying SDS directly to
Gaussian-based 3D super-resolution leads to undesirable and redundant 3D
Gaussian primitives, due to the randomness brought by generative priors. To
mitigate this issue, we introduce two simple yet effective techniques to reduce
stochastic disturbances introduced by SDS. Specifically, we 1) shrink the range
of diffusion timestep in SDS with an annealing strategy; 2) randomly discard
redundant Gaussian primitives during densification. Extensive experiments have
demonstrated that our proposed GaussainSR can attain high-quality results for
HRNVS with only low-resolution inputs on both synthetic and real-world
datasets. Project page: https://chchnii.github.io/GaussianSR/
The paper describes GaussianSR, a method leveraging 3D Gaussian Splatting and Score Distillation Sampling to achieve high-resolution novel view synthesis from low-resolution inputs, addressing data scarcity and rendering speed.
Neural Radiance Field 3D Gaussian Splatting HRNVS 2D diffusion priors Score Distillation Sampling Gaussian primitives annealing strategy