Sequence Matters: Harnessing Video Models in 3D Super-Resolution
3D super-resolution aims to reconstruct high-fidelity 3D models from
low-resolution (LR) multi-view images. Early studies primarily focused on
single-image super-resolution (SISR) models to upsample LR images into
high-resolution images. However, these methods often lack view consistency
because they operate independently on each image. Although various
post-processing techniques have been extensively explored to mitigate these
inconsistencies, they have yet to fully resolve the issues. In this paper, we
perform a comprehensive study of 3D super-resolution by leveraging video
super-resolution (VSR) models. By utilizing VSR models, we ensure a higher
degree of spatial consistency and can reference surrounding spatial
information, leading to more accurate and detailed reconstructions. Our
findings reveal that VSR models can perform remarkably well even on sequences
that lack precise spatial alignment. Given this observation, we propose a
simple yet practical approach to align LR images without involving fine-tuning
or generating 'smooth' trajectory from the trained 3D models over LR images.
The experimental results show that the surprisingly simple algorithms can
achieve the state-of-the-art results of 3D super-resolution tasks on standard
benchmark datasets, such as the NeRF-synthetic and MipNeRF-360 datasets.
Project page: https://ko-lani.github.io/Sequence-Matters
676687cb5b17ac358c9ff22b
Hyun-kyu Ko
claimed_verified
2024-12-21T15:19:33.358Z
6742e770459000b812f3a276
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/nsEgzIm_-mPQXgYXXh05k.png
Lani Ko
user
lanikoisgod
676687cb5b17ac358c9ff22c
Dongheok Park
claimed_verified
2024-12-21T15:19:35.720Z
645c7f5a7848314a460e0b69
/avatars/41eb5620dcfdbacf48b1345a987f8d8f.svg
Dongheok Park
user
HEOK
676687cb5b17ac358c9ff22d
Youngin Park
admin_assigned
2024-12-23T11:34:45.160Z
64816a98cb64a2b85a460aa3
/avatars/7a36f54647c9424e37579b56dc6b35d1.svg
Youngin Park
user
yi0109-park
676687cb5b17ac358c9ff22e
Byeonghyeon Lee
admin_assigned
2024-12-23T11:34:51.729Z
614c218e7e1c37ffc55c7510
/avatars/f86636fceb97480a73b7ca7b5e20d8f5.svg
Byeonghyeon Lee
user
blee
676687cb5b17ac358c9ff22f
Juhee Han
admin_assigned
2024-12-23T11:34:57.023Z
66036d0530394deb4ae1defd
/avatars/0037ccccb1dccb03d634e3651952fe3e.svg
Juhee Han
user
juxhee
676687cb5b17ac358c9ff230
Eunbyung Park
admin_assigned
2024-12-23T11:35:02.518Z
655e0141d36a195f663ee4b0
/avatars/97bb695ccefdcb2139b94bcae808cf99.svg
Eunbyung Park
user
epark
Sequence Matters: Harnessing Video Models in 3D Super-Resolution
3D super-resolution aims to reconstruct high-fidelity 3D models from
low-resolution (LR) multi-view images. Early studies primarily focused on
single-image super-resolution (SISR) models to upsample LR images into
high-resolution images. However, these methods often lack view consistency
because they operate independently on each image. Although various
post-processing techniques have been extensively explored to mitigate these
inconsistencies, they have yet to fully resolve the issues. In this paper, we
perform a comprehensive study of 3D super-resolution by leveraging video
super-resolution (VSR) models. By utilizing VSR models, we ensure a higher
degree of spatial consistency and can reference surrounding spatial
information, leading to more accurate and detailed reconstructions. Our
findings reveal that VSR models can perform remarkably well even on sequences
that lack precise spatial alignment. Given this observation, we propose a
simple yet practical approach to align LR images without involving fine-tuning
or generating 'smooth' trajectory from the trained 3D models over LR images.
The experimental results show that the surprisingly simple algorithms can
achieve the state-of-the-art results of 3D super-resolution tasks on standard
benchmark datasets, such as the NeRF-synthetic and MipNeRF-360 datasets.
Project page: https://ko-lani.github.io/Sequence-Matters
Video super-resolution models enhance 3D super-resolution by ensuring spatial consistency across low-resolution multi-view images without the need for complex alignment techniques.
single-image super-resolution view consistency video super-resolution spatial consistency NeRF-synthetic MipNeRF-360