PUSA V1.0: Surpassing Wan-I2V with $500 Training Cost by Vectorized
  Timestep Adaptation
The rapid advancement of video diffusion models has been hindered by
fundamental limitations in temporal modeling, particularly the rigid
synchronization of frame evolution imposed by conventional scalar timestep
variables. While task-specific adaptations and autoregressive models have
sought to address these challenges, they remain constrained by computational
inefficiency, catastrophic forgetting, or narrow applicability. In this work,
we present Pusa, a groundbreaking paradigm that leverages vectorized timestep
adaptation (VTA) to enable fine-grained temporal control within a unified video
diffusion framework. Besides, VTA is a non-destructive adaptation, which means
it fully preserves the capabilities of the base model. By finetuning the SOTA
Wan2.1-T2V-14B model with VTA, we achieve unprecedented efficiency --
surpassing the performance of Wan-I2V-14B with leq 1/200 of the training
cost (\500 vs. \geq 100,000) and leq 1/2500 of the dataset size (4K vs.
geq 10M samples). Pusa not only sets a new standard for image-to-video (I2V)
generation, achieving a VBench-I2V total score of 87.32\% (vs. 86.86\% of
Wan-I2V-14B), but also unlocks many zero-shot multi-task capabilities such as
start-end frames and video extension -- all without task-specific training.
Meanwhile, Pusa can still perform text-to-video generation. Mechanistic
analyses reveal that our approach preserves the foundation model's generative
priors while surgically injecting temporal dynamics, avoiding the combinatorial
explosion inherent to vectorized timesteps. This work establishes a scalable,
efficient, and versatile paradigm for next-generation video synthesis,
democratizing high-fidelity video generation for research and industry alike.
Code is open-sourced at https://github.com/Yaofang-Liu/Pusa-VidGen
688035ff00f4b5a05f2fbd9b
Yaofang Liu
claimed_verified
2025-07-25T08:39:17.956Z
658d46649a1397992a4d657f
/avatars/66d23095b8d0368ea862778dbf123a67.svg
Raphael Liu
user
RaphaelLiu
688035ff00f4b5a05f2fbd9c
Yumeng Ren
688035ff00f4b5a05f2fbd9d
Aitor Artola
688035ff00f4b5a05f2fbd9e
Yuxuan Hu
688035ff00f4b5a05f2fbd9f
Xiaodong Cun
688035ff00f4b5a05f2fbda0
Xiaotong Zhao
688035ff00f4b5a05f2fbda1
Alan Zhao
688035ff00f4b5a05f2fbda2
Raymond H. Chan
688035ff00f4b5a05f2fbda3
Suiyun Zhang
688035ff00f4b5a05f2fbda4
Rui Liu
688035ff00f4b5a05f2fbda5
Dandan Tu
688035ff00f4b5a05f2fbda6
Jean-Michel Morel
PUSA V1.0: Surpassing Wan-I2V with $500 Training Cost by Vectorized
  Timestep Adaptation
The rapid advancement of video diffusion models has been hindered by
fundamental limitations in temporal modeling, particularly the rigid
synchronization of frame evolution imposed by conventional scalar timestep
variables. While task-specific adaptations and autoregressive models have
sought to address these challenges, they remain constrained by computational
inefficiency, catastrophic forgetting, or narrow applicability. In this work,
we present Pusa, a groundbreaking paradigm that leverages vectorized timestep
adaptation (VTA) to enable fine-grained temporal control within a unified video
diffusion framework. Besides, VTA is a non-destructive adaptation, which means
it fully preserves the capabilities of the base model. By finetuning the SOTA
Wan2.1-T2V-14B model with VTA, we achieve unprecedented efficiency --
surpassing the performance of Wan-I2V-14B with leq 1/200 of the training
cost (\500 vs. \geq 100,000) and leq 1/2500 of the dataset size (4K vs.
geq 10M samples). Pusa not only sets a new standard for image-to-video (I2V)
generation, achieving a VBench-I2V total score of 87.32\% (vs. 86.86\% of
Wan-I2V-14B), but also unlocks many zero-shot multi-task capabilities such as
start-end frames and video extension -- all without task-specific training.
Meanwhile, Pusa can still perform text-to-video generation. Mechanistic
analyses reveal that our approach preserves the foundation model's generative
priors while surgically injecting temporal dynamics, avoiding the combinatorial
explosion inherent to vectorized timesteps. This work establishes a scalable,
efficient, and versatile paradigm for next-generation video synthesis,
democratizing high-fidelity video generation for research and industry alike.
Code is open-sourced at https://github.com/Yaofang-Liu/Pusa-VidGen
Pusa, a vectorized timestep adaptation framework, enhances video diffusion models by enabling efficient and versatile temporal control, significantly reducing training costs and dataset sizes while improving performance and enabling zero-shot multi-task capabilities.
video diffusion models temporal modeling scalar timestep variables autoregressive models vectorized timestep adaptation VTA fine-grained temporal control unified video diffusion framework Wan2.1-T2V-14B Wan-I2V-14B image-to-video generation VBench-I2V text-to-video generation generative priors temporal dynamics combinatorial explosion