DenseDPO: Fine-Grained Temporal Preference Optimization for Video
  Diffusion Models
Direct Preference Optimization (DPO) has recently been applied as a
post-training technique for text-to-video diffusion models. To obtain training
data, annotators are asked to provide preferences between two videos generated
from independent noise. However, this approach prohibits fine-grained
comparisons, and we point out that it biases the annotators towards low-motion
clips as they often contain fewer visual artifacts. In this work, we introduce
DenseDPO, a method that addresses these shortcomings by making three
contributions. First, we create each video pair for DPO by denoising corrupted
copies of a ground truth video. This results in aligned pairs with similar
motion structures while differing in local details, effectively neutralizing
the motion bias. Second, we leverage the resulting temporal alignment to label
preferences on short segments rather than entire clips, yielding a denser and
more precise learning signal. With only one-third of the labeled data, DenseDPO
greatly improves motion generation over vanilla DPO, while matching it in text
alignment, visual quality, and temporal consistency. Finally, we show that
DenseDPO unlocks automatic preference annotation using off-the-shelf Vision
Language Models (VLMs): GPT accurately predicts segment-level preferences
similar to task-specifically fine-tuned video reward models, and DenseDPO
trained on these labels achieves performance close to using human labels.
68412f853c22997c7329f3a0
Ziyi Wu
claimed_verified
2025-06-05T07:26:43.314Z
62980664ff0acd7e027d6686
/avatars/364d4c8432c24775a099641fc576dbdc.svg
Ziyi Wu
user
Dazitu616
68412f853c22997c7329f3a1
Anil Kag
admin_assigned
2025-06-05T12:55:11.687Z
66b01ee8e53bbad918362856
/avatars/293529589a91dd7a95909d66727db224.svg
Anil Kag
user
anilkagak2
68412f853c22997c7329f3a2
Ivan Skorokhodov
admin_assigned
2025-06-05T12:55:18.188Z
63610db13c7147fae7de88e3
/avatars/d7e97a16cfee39e1e50d7a5b747876f1.svg
Ivan Skorokhodov
user
universome
68412f853c22997c7329f3a3
Willi Menapace
admin_assigned
2025-06-05T12:55:30.315Z
6315358a362e3e95ea538081
/avatars/3b089a25a87c2e83c6b23ccb5d2dc73e.svg
Willi Menapace
user
willi-menapace
68412f853c22997c7329f3a4
Ashkan Mirzaei
admin_assigned
2025-06-05T12:55:36.943Z
634db15dd00bb5d92c3bd94f
/avatars/9a6a1231bc5205911272d83527593f1a.svg
Ashkan Mirzaei
user
ashmrz
68412f853c22997c7329f3a5
Igor Gilitschenski
admin_assigned
2025-06-05T12:55:42.860Z
64893f0b9ab28735f1a1ec0b
https://cdn-avatars.huggingface.co/v1/production/uploads/64893f0b9ab28735f1a1ec0b/2lPBjV16l2qLEWOKU2zwR.jpeg
Igor Gilitschenski
user
igilitschenski
68412f853c22997c7329f3a6
Sergey Tulyakov
68412f853c22997c7329f3a7
Aliaksandr Siarohin
admin_assigned
2025-06-05T12:55:52.883Z
64276311eb9a0ed86180715b
/avatars/76f933cd549f10e5e2db379de235d304.svg
Aliaksandr Siarohin
user
aliaksandr-siarohin
DenseDPO: Fine-Grained Temporal Preference Optimization for Video
  Diffusion Models
Direct Preference Optimization (DPO) has recently been applied as a
post-training technique for text-to-video diffusion models. To obtain training
data, annotators are asked to provide preferences between two videos generated
from independent noise. However, this approach prohibits fine-grained
comparisons, and we point out that it biases the annotators towards low-motion
clips as they often contain fewer visual artifacts. In this work, we introduce
DenseDPO, a method that addresses these shortcomings by making three
contributions. First, we create each video pair for DPO by denoising corrupted
copies of a ground truth video. This results in aligned pairs with similar
motion structures while differing in local details, effectively neutralizing
the motion bias. Second, we leverage the resulting temporal alignment to label
preferences on short segments rather than entire clips, yielding a denser and
more precise learning signal. With only one-third of the labeled data, DenseDPO
greatly improves motion generation over vanilla DPO, while matching it in text
alignment, visual quality, and temporal consistency. Finally, we show that
DenseDPO unlocks automatic preference annotation using off-the-shelf Vision
Language Models (VLMs): GPT accurately predicts segment-level preferences
similar to task-specifically fine-tuned video reward models, and DenseDPO
trained on these labels achieves performance close to using human labels.