Self Forcing: Bridging the Train-Test Gap in Autoregressive Video
  Diffusion
We introduce Self Forcing, a novel training paradigm for autoregressive video
diffusion models. It addresses the longstanding issue of exposure bias, where
models trained on ground-truth context must generate sequences conditioned on
their own imperfect outputs during inference. Unlike prior methods that denoise
future frames based on ground-truth context frames, Self Forcing conditions
each frame's generation on previously self-generated outputs by performing
autoregressive rollout with key-value (KV) caching during training. This
strategy enables supervision through a holistic loss at the video level that
directly evaluates the quality of the entire generated sequence, rather than
relying solely on traditional frame-wise objectives. To ensure training
efficiency, we employ a few-step diffusion model along with a stochastic
gradient truncation strategy, effectively balancing computational cost and
performance. We further introduce a rolling KV cache mechanism that enables
efficient autoregressive video extrapolation. Extensive experiments demonstrate
that our approach achieves real-time streaming video generation with sub-second
latency on a single GPU, while matching or even surpassing the generation
quality of significantly slower and non-causal diffusion models. Project
website: http://self-forcing.github.io/
68485e5b4fe3b60e21b258bd
Xun Huang
claimed_verified
2025-06-12T12:42:51.298Z
6732bfed9e05f3e257315c8e
/avatars/abe9aa6914feef77b7205fe030e5bcc5.svg
Xun Huang
user
xunhuang1995
68485e5b4fe3b60e21b258be
Zhengqi Li
68485e5b4fe3b60e21b258bf
Guande He
claimed_verified
2025-06-11T08:35:40.950Z
67492ee82ad3cfc108a41bbb
/avatars/7ad03e55a8791c62f1271a5c9bf8cc60.svg
Guande He
user
gdhe17
68485e5b4fe3b60e21b258c0
Mingyuan Zhou
68485e5b4fe3b60e21b258c1
Eli Shechtman
Self Forcing: Bridging the Train-Test Gap in Autoregressive Video
  Diffusion
We introduce Self Forcing, a novel training paradigm for autoregressive video
diffusion models. It addresses the longstanding issue of exposure bias, where
models trained on ground-truth context must generate sequences conditioned on
their own imperfect outputs during inference. Unlike prior methods that denoise
future frames based on ground-truth context frames, Self Forcing conditions
each frame's generation on previously self-generated outputs by performing
autoregressive rollout with key-value (KV) caching during training. This
strategy enables supervision through a holistic loss at the video level that
directly evaluates the quality of the entire generated sequence, rather than
relying solely on traditional frame-wise objectives. To ensure training
efficiency, we employ a few-step diffusion model along with a stochastic
gradient truncation strategy, effectively balancing computational cost and
performance. We further introduce a rolling KV cache mechanism that enables
efficient autoregressive video extrapolation. Extensive experiments demonstrate
that our approach achieves real-time streaming video generation with sub-second
latency on a single GPU, while matching or even surpassing the generation
quality of significantly slower and non-causal diffusion models. Project
website: http://self-forcing.github.io/
Self Forcing, a novel training method for autoregressive video diffusion models, reduces exposure bias and improves generation quality through holistic video-level supervision and efficient caching mechanisms.
Self Forcing autoregressive video diffusion models exposure bias denoising key-value (KV) caching autoregressive rollout holistic loss few-step diffusion model stochastic gradient truncation rolling KV cache mechanism video extrapolation