TPDiff: Temporal Pyramid Video Diffusion Model
The development of video diffusion models unveils a significant challenge:
the substantial computational demands. To mitigate this challenge, we note that
the reverse process of diffusion exhibits an inherent entropy-reducing nature.
Given the inter-frame redundancy in video modality, maintaining full frame
rates in high-entropy stages is unnecessary. Based on this insight, we propose
TPDiff, a unified framework to enhance training and inference efficiency. By
dividing diffusion into several stages, our framework progressively increases
frame rate along the diffusion process with only the last stage operating on
full frame rate, thereby optimizing computational efficiency. To train the
multi-stage diffusion model, we introduce a dedicated training framework:
stage-wise diffusion. By solving the partitioned probability flow ordinary
differential equations (ODE) of diffusion under aligned data and noise, our
training strategy is applicable to various diffusion forms and further enhances
training efficiency. Comprehensive experimental evaluations validate the
generality of our method, demonstrating 50% reduction in training cost and 1.5x
improvement in inference efficiency.
67d274c467e782a7eeb4ab70
Lingmin Ran
admin_assigned
2025-03-14T11:19:50.355Z
63fb063d1b4b1bd4e7fb3c86
/avatars/241e3b1340bf6450893a821ae73f2f56.svg
Andy
user
Lingmin-Ran
67d274c467e782a7eeb4ab71
Mike Zheng Shou
admin_assigned
2025-03-14T11:19:35.941Z
661ab3da2b14565c7acccf5c
/avatars/fa4fc03664803e02aede4d4c3d50b393.svg
Mike Zheng Shou
user
AnalMom
TPDiff: Temporal Pyramid Video Diffusion Model
The development of video diffusion models unveils a significant challenge:
the substantial computational demands. To mitigate this challenge, we note that
the reverse process of diffusion exhibits an inherent entropy-reducing nature.
Given the inter-frame redundancy in video modality, maintaining full frame
rates in high-entropy stages is unnecessary. Based on this insight, we propose
TPDiff, a unified framework to enhance training and inference efficiency. By
dividing diffusion into several stages, our framework progressively increases
frame rate along the diffusion process with only the last stage operating on
full frame rate, thereby optimizing computational efficiency. To train the
multi-stage diffusion model, we introduce a dedicated training framework:
stage-wise diffusion. By solving the partitioned probability flow ordinary
differential equations (ODE) of diffusion under aligned data and noise, our
training strategy is applicable to various diffusion forms and further enhances
training efficiency. Comprehensive experimental evaluations validate the
generality of our method, demonstrating 50% reduction in training cost and 1.5x
improvement in inference efficiency.
A multi-stage diffusion framework, TPDiff, enhances video diffusion model efficiency by reducing full frame rate during high-entropy stages, leading to diminished training costs and improved inference efficiency.
video diffusion models entropy-reducing inter-frame redundancy frame rate stage-wise diffusion probability flow ordinary differential equations diffusion forms training efficiency inference efficiency