SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video
  Restoration
Video restoration poses non-trivial challenges in maintaining fidelity while
recovering temporally consistent details from unknown degradations in the wild.
Despite recent advances in diffusion-based restoration, these methods often
face limitations in generation capability and sampling efficiency. In this
work, we present SeedVR, a diffusion transformer designed to handle real-world
video restoration with arbitrary length and resolution. The core design of
SeedVR lies in the shifted window attention that facilitates effective
restoration on long video sequences. SeedVR further supports variable-sized
windows near the boundary of both spatial and temporal dimensions, overcoming
the resolution constraints of traditional window attention. Equipped with
contemporary practices, including causal video autoencoder, mixed image and
video training, and progressive training, SeedVR achieves highly-competitive
performance on both synthetic and real-world benchmarks, as well as
AI-generated videos. Extensive experiments demonstrate SeedVR's superiority
over existing methods for generic video restoration.
67779a845d0e6d1912b339a7
Jianyi Wang
claimed_verified
2025-01-05T23:02:39.129Z
63043db17373aacccd89f49d
https://cdn-avatars.huggingface.co/v1/production/uploads/63043db17373aacccd89f49d/jzP_fPCFXeYJvAD8uA_N7.jpeg
JIANYI WANG
user
Iceclear
67779a845d0e6d1912b339a8
Zhijie Lin
67779a845d0e6d1912b339a9
Meng Wei
67779a845d0e6d1912b339aa
Yang Zhao
67779a845d0e6d1912b339ab
Ceyuan Yang
67779a845d0e6d1912b339ac
Chen Change Loy
67779a845d0e6d1912b339ad
Lu Jiang
SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video
  Restoration
Video restoration poses non-trivial challenges in maintaining fidelity while
recovering temporally consistent details from unknown degradations in the wild.
Despite recent advances in diffusion-based restoration, these methods often
face limitations in generation capability and sampling efficiency. In this
work, we present SeedVR, a diffusion transformer designed to handle real-world
video restoration with arbitrary length and resolution. The core design of
SeedVR lies in the shifted window attention that facilitates effective
restoration on long video sequences. SeedVR further supports variable-sized
windows near the boundary of both spatial and temporal dimensions, overcoming
the resolution constraints of traditional window attention. Equipped with
contemporary practices, including causal video autoencoder, mixed image and
video training, and progressive training, SeedVR achieves highly-competitive
performance on both synthetic and real-world benchmarks, as well as
AI-generated videos. Extensive experiments demonstrate SeedVR's superiority
over existing methods for generic video restoration.
SeedVR, a diffusion transformer with shifted window attention, effectively handles real-world video restoration with high performance on synthetic and real-world benchmarks.
diffusion transformer shifted window attention causal video autoencoder mixed image and video training progressive training video restoration