OmniTry: Virtual Try-On Anything without Masks
Virtual Try-ON (VTON) is a practical and widely-applied task, for which most
of existing works focus on clothes. This paper presents OmniTry, a unified
framework that extends VTON beyond garment to encompass any wearable objects,
e.g., jewelries and accessories, with mask-free setting for more practical
application. When extending to various types of objects, data curation is
challenging for obtaining paired images, i.e., the object image and the
corresponding try-on result. To tackle this problem, we propose a two-staged
pipeline: For the first stage, we leverage large-scale unpaired images, i.e.,
portraits with any wearable items, to train the model for mask-free
localization. Specifically, we repurpose the inpainting model to automatically
draw objects in suitable positions given an empty mask. For the second stage,
the model is further fine-tuned with paired images to transfer the consistency
of object appearance. We observed that the model after the first stage shows
quick convergence even with few paired samples. OmniTry is evaluated on a
comprehensive benchmark consisting of 12 common classes of wearable objects,
with both in-shop and in-the-wild images. Experimental results suggest that
OmniTry shows better performance on both object localization and
ID-preservation compared with existing methods. The code, model weights, and
evaluation benchmark of OmniTry will be made publicly available at
https://omnitry.github.io/.
68a533b66cf0bf898542ec55
Yutong Feng
claimed_verified
2025-08-20T08:51:41.803Z
64a54e468cfaa458bd6844bf
https://cdn-avatars.huggingface.co/v1/production/uploads/64a54e468cfaa458bd6844bf/5Gmf4tAr59GNl-2VaZDbu.png
Yutong Feng
user
fengyutong
68a533b66cf0bf898542ec56
Linlin Zhang
68a533b66cf0bf898542ec57
Hengyuan Cao
68a533b66cf0bf898542ec58
Yiming Chen
68a533b66cf0bf898542ec59
Xiaoduan Feng
68a533b66cf0bf898542ec5a
Jian Cao
68a533b66cf0bf898542ec5b
Yuxiong Wu
68a533b66cf0bf898542ec5c
Bin Wang
OmniTry: Virtual Try-On Anything without Masks
Virtual Try-ON (VTON) is a practical and widely-applied task, for which most
of existing works focus on clothes. This paper presents OmniTry, a unified
framework that extends VTON beyond garment to encompass any wearable objects,
e.g., jewelries and accessories, with mask-free setting for more practical
application. When extending to various types of objects, data curation is
challenging for obtaining paired images, i.e., the object image and the
corresponding try-on result. To tackle this problem, we propose a two-staged
pipeline: For the first stage, we leverage large-scale unpaired images, i.e.,
portraits with any wearable items, to train the model for mask-free
localization. Specifically, we repurpose the inpainting model to automatically
draw objects in suitable positions given an empty mask. For the second stage,
the model is further fine-tuned with paired images to transfer the consistency
of object appearance. We observed that the model after the first stage shows
quick convergence even with few paired samples. OmniTry is evaluated on a
comprehensive benchmark consisting of 12 common classes of wearable objects,
with both in-shop and in-the-wild images. Experimental results suggest that
OmniTry shows better performance on both object localization and
ID-preservation compared with existing methods. The code, model weights, and
evaluation benchmark of OmniTry will be made publicly available at
https://omnitry.github.io/.
OmniTry extends Virtual Try-ON to various wearable objects using a two-stage pipeline that combines unpaired and paired image training to improve object localization and appearance consistency.
Virtual Try-ON OmniTry mask-free localization inpainting model fine-tuning object localization ID-preservation