Dress&Dance: Dress up and Dance as You Like It - Technical Preview
We present Dress&Dance, a video diffusion framework that generates high
quality 5-second-long 24 FPS virtual try-on videos at 1152x720 resolution of a
user wearing desired garments while moving in accordance with a given reference
video. Our approach requires a single user image and supports a range of tops,
bottoms, and one-piece garments, as well as simultaneous tops and bottoms
try-on in a single pass. Key to our framework is CondNet, a novel conditioning
network that leverages attention to unify multi-modal inputs (text, images, and
videos), thereby enhancing garment registration and motion fidelity. CondNet is
trained on heterogeneous training data, combining limited video data and a
larger, more readily available image dataset, in a multistage progressive
manner. Dress&Dance outperforms existing open source and commercial solutions
and enables a high quality and flexible try-on experience.
68b12ba4b19c5400014849d5
Jun-Kun Chen
68b12ba4b19c5400014849d6
Aayush Bansal
68b12ba4b19c5400014849d7
Minh Phuoc Vo
68b12ba4b19c5400014849d8
Yu-Xiong Wang
Dress&Dance: Dress up and Dance as You Like It - Technical Preview
We present Dress&Dance, a video diffusion framework that generates high
quality 5-second-long 24 FPS virtual try-on videos at 1152x720 resolution of a
user wearing desired garments while moving in accordance with a given reference
video. Our approach requires a single user image and supports a range of tops,
bottoms, and one-piece garments, as well as simultaneous tops and bottoms
try-on in a single pass. Key to our framework is CondNet, a novel conditioning
network that leverages attention to unify multi-modal inputs (text, images, and
videos), thereby enhancing garment registration and motion fidelity. CondNet is
trained on heterogeneous training data, combining limited video data and a
larger, more readily available image dataset, in a multistage progressive
manner. Dress&Dance outperforms existing open source and commercial solutions
and enables a high quality and flexible try-on experience.
A video diffusion framework generates high-quality virtual try-on videos using a conditioning network that unifies multi-modal inputs, outperforming existing solutions.
video diffusion CondNet attention multi-modal inputs heterogeneous training data multistage progressive training