FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting
Novel view synthesis from limited observations remains an important and
persistent task. However, high efficiency in existing NeRF-based few-shot view
synthesis is often compromised to obtain an accurate 3D representation. To
address this challenge, we propose a few-shot view synthesis framework based on
3D Gaussian Splatting that enables real-time and photo-realistic view synthesis
with as few as three training views. The proposed method, dubbed FSGS, handles
the extremely sparse initialized SfM points with a thoughtfully designed
Gaussian Unpooling process. Our method iteratively distributes new Gaussians
around the most representative locations, subsequently infilling local details
in vacant areas. We also integrate a large-scale pre-trained monocular depth
estimator within the Gaussians optimization process, leveraging online
augmented views to guide the geometric optimization towards an optimal
solution. Starting from sparse points observed from limited input viewpoints,
our FSGS can accurately grow into unseen regions, comprehensively covering the
scene and boosting the rendering quality of novel views. Overall, FSGS achieves
state-of-the-art performance in both accuracy and rendering efficiency across
diverse datasets, including LLFF, Mip-NeRF360, and Blender. Project website:
https://zehaozhu.github.io/FSGS/.
656db8e1ca3e0756db3362ad
Zehao Zhu
admin_assigned
2023-12-04T14:37:22.572Z
63611b5b6bd72c97d0fe5dc8
/avatars/8905d4c0a930c7ba6e057c8b01f22a2e.svg
Zehao Zhu
user
zzhzzh
656db8e1ca3e0756db3362ae
Zhiwen Fan
admin_assigned
2023-12-04T14:37:34.011Z
6526386e1c6a09292d8d0a22
/avatars/471de830de2d775d35368678c1579f87.svg
fan
user
Fanzhiwen
656db8e1ca3e0756db3362af
Yifan Jiang
656db8e1ca3e0756db3362b0
Zhangyang Wang
FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting
Novel view synthesis from limited observations remains an important and
persistent task. However, high efficiency in existing NeRF-based few-shot view
synthesis is often compromised to obtain an accurate 3D representation. To
address this challenge, we propose a few-shot view synthesis framework based on
3D Gaussian Splatting that enables real-time and photo-realistic view synthesis
with as few as three training views. The proposed method, dubbed FSGS, handles
the extremely sparse initialized SfM points with a thoughtfully designed
Gaussian Unpooling process. Our method iteratively distributes new Gaussians
around the most representative locations, subsequently infilling local details
in vacant areas. We also integrate a large-scale pre-trained monocular depth
estimator within the Gaussians optimization process, leveraging online
augmented views to guide the geometric optimization towards an optimal
solution. Starting from sparse points observed from limited input viewpoints,
our FSGS can accurately grow into unseen regions, comprehensively covering the
scene and boosting the rendering quality of novel views. Overall, FSGS achieves
state-of-the-art performance in both accuracy and rendering efficiency across
diverse datasets, including LLFF, Mip-NeRF360, and Blender. Project website:
https://zehaozhu.github.io/FSGS/.
FSGS, a few-shot view synthesis framework using 3D Gaussian Splatting, achieves real-time photo-realistic rendering with three training views and state-of-the-art accuracy and efficiency.
NeRF few-shot view synthesis 3D Gaussian Splatting Gaussian Unpooling monocular depth estimator geometric optimization rendering quality LLFF Mip-NeRF360 Blender