FitDiT: Advancing the Authentic Garment Details for High-fidelity
  Virtual Try-on
Although image-based virtual try-on has made considerable progress, emerging
approaches still encounter challenges in producing high-fidelity and robust
fitting images across diverse scenarios. These methods often struggle with
issues such as texture-aware maintenance and size-aware fitting, which hinder
their overall effectiveness. To address these limitations, we propose a novel
garment perception enhancement technique, termed FitDiT, designed for
high-fidelity virtual try-on using Diffusion Transformers (DiT) allocating more
parameters and attention to high-resolution features. First, to further improve
texture-aware maintenance, we introduce a garment texture extractor that
incorporates garment priors evolution to fine-tune garment feature,
facilitating to better capture rich details such as stripes, patterns, and
text. Additionally, we introduce frequency-domain learning by customizing a
frequency distance loss to enhance high-frequency garment details. To tackle
the size-aware fitting issue, we employ a dilated-relaxed mask strategy that
adapts to the correct length of garments, preventing the generation of garments
that fill the entire mask area during cross-category try-on. Equipped with the
above design, FitDiT surpasses all baselines in both qualitative and
quantitative evaluations. It excels in producing well-fitting garments with
photorealistic and intricate details, while also achieving competitive
inference times of 4.57 seconds for a single 1024x768 image after DiT structure
slimming, outperforming existing methods.
673c23932b49c38cbe85be38
Boyuan Jiang
admin_assigned
2024-11-19T16:38:21.750Z
63b63a470d5913eee48bc2b3
/avatars/9be7857a6e06d6cf99bd2829ded9ef02.svg
jiang
user
BoyuanJiang
673c23932b49c38cbe85be39
Xiaobin Hu
673c23932b49c38cbe85be3a
Donghao Luo
673c23932b49c38cbe85be3b
Qingdong He
admin_assigned
2024-11-19T16:38:47.038Z
64799de9587656b38e8dcf7a
/avatars/87069f03d44cc2d9e0458bc17bd0159a.svg
heqingdong
user
hqd
673c23932b49c38cbe85be3c
Chengming Xu
admin_assigned
2024-11-19T16:38:52.876Z
652fab9d04a34a9282bf29d6
/avatars/cd5967b37ebb1225e9ae1d46f196e2e2.svg
Chengming Xu
user
ChengmingX
673c23932b49c38cbe85be3d
Jinlong Peng
673c23932b49c38cbe85be3e
Jiangning Zhang
673c23932b49c38cbe85be3f
Chengjie Wang
673c23932b49c38cbe85be40
Yunsheng Wu
673c23932b49c38cbe85be41
Yanwei Fu
FitDiT: Advancing the Authentic Garment Details for High-fidelity
  Virtual Try-on
Although image-based virtual try-on has made considerable progress, emerging
approaches still encounter challenges in producing high-fidelity and robust
fitting images across diverse scenarios. These methods often struggle with
issues such as texture-aware maintenance and size-aware fitting, which hinder
their overall effectiveness. To address these limitations, we propose a novel
garment perception enhancement technique, termed FitDiT, designed for
high-fidelity virtual try-on using Diffusion Transformers (DiT) allocating more
parameters and attention to high-resolution features. First, to further improve
texture-aware maintenance, we introduce a garment texture extractor that
incorporates garment priors evolution to fine-tune garment feature,
facilitating to better capture rich details such as stripes, patterns, and
text. Additionally, we introduce frequency-domain learning by customizing a
frequency distance loss to enhance high-frequency garment details. To tackle
the size-aware fitting issue, we employ a dilated-relaxed mask strategy that
adapts to the correct length of garments, preventing the generation of garments
that fill the entire mask area during cross-category try-on. Equipped with the
above design, FitDiT surpasses all baselines in both qualitative and
quantitative evaluations. It excels in producing well-fitting garments with
photorealistic and intricate details, while also achieving competitive
inference times of 4.57 seconds for a single 1024x768 image after DiT structure
slimming, outperforming existing methods.
FitDiT, a novel garment perception enhancement technique using Diffusion Transformers, addresses texture-aware maintenance and size-aware fitting in high-fidelity virtual try-on.
Diffusion Transformers (DiT) garment perception enhancement texture-aware maintenance garment texture extractor garment priors evolution frequency-domain learning frequency distance loss size-aware fitting dilated-relaxed mask strategy well-fitting garments photorealistic details