DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a
  Stage-Wise Diffusion Transformer Framework
Video virtual try-on (VVT) technology has garnered considerable academic
interest owing to its promising applications in e-commerce advertising and
entertainment. However, most existing end-to-end methods rely heavily on scarce
paired garment-centric datasets and fail to effectively leverage priors of
advanced visual models and test-time inputs, making it challenging to
accurately preserve fine-grained garment details and maintain temporal
consistency in unconstrained scenarios. To address these challenges, we propose
DreamVVT, a carefully designed two-stage framework built upon Diffusion
Transformers (DiTs), which is inherently capable of leveraging diverse unpaired
human-centric data to enhance adaptability in real-world scenarios. To further
leverage prior knowledge from pretrained models and test-time inputs, in the
first stage, we sample representative frames from the input video and utilize a
multi-frame try-on model integrated with a vision-language model (VLM), to
synthesize high-fidelity and semantically consistent keyframe try-on images.
These images serve as complementary appearance guidance for subsequent video
generation. In the second stage, skeleton maps together with
fine-grained motion and appearance descriptions are extracted from the input
content, and these along with the keyframe try-on images are then fed into a
pretrained video generation model enhanced with LoRA adapters. This ensures
long-term temporal coherence for unseen regions and enables highly plausible
dynamic motions. Extensive quantitative and qualitative experiments demonstrate
that DreamVVT surpasses existing methods in preserving detailed garment content
and temporal stability in real-world scenarios. Our project page
https://virtu-lab.github.io/
68944b9e741a16f544fbcfec
Tongchun Zuo
68944b9e741a16f544fbcfed
Zaiyu Huang
68944b9e741a16f544fbcfee
Shuliang Ning
68944b9e741a16f544fbcfef
Ente Lin
68944b9e741a16f544fbcff0
Chao Liang
claimed_verified
2025-08-08T16:21:50.866Z
66d7ca4c0429a62c38c46bf5
https://cdn-avatars.huggingface.co/v1/production/uploads/66d7ca4c0429a62c38c46bf5/N3puSDlwFA8WOxjHjObrj.jpeg
liangdebugger
user
chao0412
68944b9e741a16f544fbcff1
Zerong Zheng
68944b9e741a16f544fbcff2
Jianwen Jiang
68944b9e741a16f544fbcff3
Yuan Zhang
68944b9e741a16f544fbcff4
Mingyuan Gao
68944b9e741a16f544fbcff5
Xin Dong
DreamVVT: Mastering Realistic Video Virtual Try-On in the Wild via a
  Stage-Wise Diffusion Transformer Framework
Video virtual try-on (VVT) technology has garnered considerable academic
interest owing to its promising applications in e-commerce advertising and
entertainment. However, most existing end-to-end methods rely heavily on scarce
paired garment-centric datasets and fail to effectively leverage priors of
advanced visual models and test-time inputs, making it challenging to
accurately preserve fine-grained garment details and maintain temporal
consistency in unconstrained scenarios. To address these challenges, we propose
DreamVVT, a carefully designed two-stage framework built upon Diffusion
Transformers (DiTs), which is inherently capable of leveraging diverse unpaired
human-centric data to enhance adaptability in real-world scenarios. To further
leverage prior knowledge from pretrained models and test-time inputs, in the
first stage, we sample representative frames from the input video and utilize a
multi-frame try-on model integrated with a vision-language model (VLM), to
synthesize high-fidelity and semantically consistent keyframe try-on images.
These images serve as complementary appearance guidance for subsequent video
generation. In the second stage, skeleton maps together with
fine-grained motion and appearance descriptions are extracted from the input
content, and these along with the keyframe try-on images are then fed into a
pretrained video generation model enhanced with LoRA adapters. This ensures
long-term temporal coherence for unseen regions and enables highly plausible
dynamic motions. Extensive quantitative and qualitative experiments demonstrate
that DreamVVT surpasses existing methods in preserving detailed garment content
and temporal stability in real-world scenarios. Our project page
https://virtu-lab.github.io/
DreamVVT, a two-stage framework using Diffusion Transformers and LoRA adapters, enhances video virtual try-on by leveraging unpaired human-centric data and pretrained models to preserve garment details and temporal consistency.
Diffusion Transformers DiTs multi-frame try-on model vision-language model VLM skeleton maps fine-grained motion appearance descriptions pretrained video generation model LoRA adapters long-term temporal coherence