Spatiotemporal Skip Guidance for Enhanced Video Diffusion Sampling
Diffusion models have emerged as a powerful tool for generating high-quality
images, videos, and 3D content. While sampling guidance techniques like CFG
improve quality, they reduce diversity and motion. Autoguidance mitigates these
issues but demands extra weak model training, limiting its practicality for
large-scale models. In this work, we introduce Spatiotemporal Skip Guidance
(STG), a simple training-free sampling guidance method for enhancing
transformer-based video diffusion models. STG employs an implicit weak model
via self-perturbation, avoiding the need for external models or additional
training. By selectively skipping spatiotemporal layers, STG produces an
aligned, degraded version of the original model to boost sample quality without
compromising diversity or dynamic degree. Our contributions include: (1)
introducing STG as an efficient, high-performing guidance technique for video
diffusion models, (2) eliminating the need for auxiliary models by simulating a
weak model through layer skipping, and (3) ensuring quality-enhanced guidance
without compromising sample diversity or dynamics unlike CFG. For additional
results, visit https://junhahyung.github.io/STGuidance.
674dcfe2aef5b4ffeca9afb2
Junha Hyung
674dcfe2aef5b4ffeca9afb3
Kinam Kim
claimed_verified
2024-12-02T16:11:06.066Z
64797735a68454566356b708
/avatars/3424d022dd8ad29b56eb41814c5c3dee.svg
Kinam Kim
user
kinam0252
674dcfe2aef5b4ffeca9afb4
Susung Hong
674dcfe2aef5b4ffeca9afb5
Min-Jung Kim
674dcfe2aef5b4ffeca9afb6
Jaegul Choo
Spatiotemporal Skip Guidance for Enhanced Video Diffusion Sampling
Diffusion models have emerged as a powerful tool for generating high-quality
images, videos, and 3D content. While sampling guidance techniques like CFG
improve quality, they reduce diversity and motion. Autoguidance mitigates these
issues but demands extra weak model training, limiting its practicality for
large-scale models. In this work, we introduce Spatiotemporal Skip Guidance
(STG), a simple training-free sampling guidance method for enhancing
transformer-based video diffusion models. STG employs an implicit weak model
via self-perturbation, avoiding the need for external models or additional
training. By selectively skipping spatiotemporal layers, STG produces an
aligned, degraded version of the original model to boost sample quality without
compromising diversity or dynamic degree. Our contributions include: (1)
introducing STG as an efficient, high-performing guidance technique for video
diffusion models, (2) eliminating the need for auxiliary models by simulating a
weak model through layer skipping, and (3) ensuring quality-enhanced guidance
without compromising sample diversity or dynamics unlike CFG. For additional
results, visit https://junhahyung.github.io/STGuidance.
Spatiotemporal Skip Guidance enhances video diffusion models by using layer skipping to achieve high sample quality without compromising diversity or dynamics.
diffusion models sampling guidance CFG autoguidance transformer-based spatiotemporal layers weak model layer skipping sample quality sample diversity dynamics