WithAnyone: Towards Controllable and ID Consistent Image Generation
Identity-consistent generation has become an important focus in text-to-image
research, with recent models achieving notable success in producing images
aligned with a reference identity. Yet, the scarcity of large-scale paired
datasets containing multiple images of the same individual forces most
approaches to adopt reconstruction-based training. This reliance often leads to
a failure mode we term copy-paste, where the model directly replicates the
reference face rather than preserving identity across natural variations in
pose, expression, or lighting. Such over-similarity undermines controllability
and limits the expressive power of generation. To address these limitations, we
(1) construct a large-scale paired dataset MultiID-2M, tailored for
multi-person scenarios, providing diverse references for each identity; (2)
introduce a benchmark that quantifies both copy-paste artifacts and the
trade-off between identity fidelity and variation; and (3) propose a novel
training paradigm with a contrastive identity loss that leverages paired data
to balance fidelity with diversity. These contributions culminate in
WithAnyone, a diffusion-based model that effectively mitigates copy-paste while
preserving high identity similarity. Extensive qualitative and quantitative
experiments demonstrate that WithAnyone significantly reduces copy-paste
artifacts, improves controllability over pose and expression, and maintains
strong perceptual quality. User studies further validate that our method
achieves high identity fidelity while enabling expressive controllable
generation.
68f1a2616e0bef323a68fcd1
Hengyuan Xu
claimed_verified
2025-10-17T05:56:54.829Z
634bde123d11eaedd889e277
https://cdn-avatars.huggingface.co/v1/production/uploads/1665916392312-noauth.png
Hengyuan Xu
user
DobyXu
68f1a2616e0bef323a68fcd2
Wei Cheng
claimed_verified
2025-10-17T04:11:13.677Z
64b914c8ace99c0723ad83a9
https://cdn-avatars.huggingface.co/v1/production/uploads/64b914c8ace99c0723ad83a9/B4gxNByeVY_xaOcjwiN1j.jpeg
Wei Cheng
user
wchengad
68f1a2616e0bef323a68fcd3
Peng Xing
claimed_verified
2025-10-18T04:49:44.189Z
649f7b742bf2e21d955c1067
/avatars/270026405d5640886a86f961a001b057.svg
xing
user
xingpng
68f1a2616e0bef323a68fcd4
Yixiao Fang
claimed_verified
2025-10-17T04:11:11.270Z
647469b9a51711a3b58bda2b
https://cdn-avatars.huggingface.co/v1/production/uploads/647469b9a51711a3b58bda2b/yeDf8Sa8IDEQyney1dGC9.jpeg
Yixiao Fang
user
fangyixiao
68f1a2616e0bef323a68fcd5
Shuhan Wu
68f1a2616e0bef323a68fcd6
Rui Wang
68f1a2616e0bef323a68fcd7
Xianfang Zeng
68f1a2616e0bef323a68fcd8
Daxin Jiang
68f1a2616e0bef323a68fcd9
Gang Yu
claimed_verified
2025-10-18T04:49:46.564Z
63417332c5565a4b8d43a0d8
https://cdn-avatars.huggingface.co/v1/production/uploads/63417332c5565a4b8d43a0d8/MmnYG7Wu2Z_lqCBvTfJmy.png
Gang Yu
user
skicy
68f1a2616e0bef323a68fcda
Xingjun Ma
68f1a2616e0bef323a68fcdb
Yu-Gang Jiang
WithAnyone: Towards Controllable and ID Consistent Image Generation
Identity-consistent generation has become an important focus in text-to-image
research, with recent models achieving notable success in producing images
aligned with a reference identity. Yet, the scarcity of large-scale paired
datasets containing multiple images of the same individual forces most
approaches to adopt reconstruction-based training. This reliance often leads to
a failure mode we term copy-paste, where the model directly replicates the
reference face rather than preserving identity across natural variations in
pose, expression, or lighting. Such over-similarity undermines controllability
and limits the expressive power of generation. To address these limitations, we
(1) construct a large-scale paired dataset MultiID-2M, tailored for
multi-person scenarios, providing diverse references for each identity; (2)
introduce a benchmark that quantifies both copy-paste artifacts and the
trade-off between identity fidelity and variation; and (3) propose a novel
training paradigm with a contrastive identity loss that leverages paired data
to balance fidelity with diversity. These contributions culminate in
WithAnyone, a diffusion-based model that effectively mitigates copy-paste while
preserving high identity similarity. Extensive qualitative and quantitative
experiments demonstrate that WithAnyone significantly reduces copy-paste
artifacts, improves controllability over pose and expression, and maintains
strong perceptual quality. User studies further validate that our method
achieves high identity fidelity while enabling expressive controllable
generation.
A diffusion-based model addresses copy-paste artifacts in text-to-image generation by using a large-scale paired dataset and a contrastive identity loss to balance identity fidelity and variation.
identity-consistent generation text-to-image reconstruction-based training copy-paste MultiID-2M contrastive identity loss diffusion-based model identity fidelity variation perceptual quality