UP2You: Fast Reconstruction of Yourself from Unconstrained Photo
  Collections
We present UP2You, the first tuning-free solution for reconstructing
high-fidelity 3D clothed portraits from extremely unconstrained in-the-wild 2D
photos. Unlike previous approaches that require "clean" inputs (e.g., full-body
images with minimal occlusions, or well-calibrated cross-view captures), UP2You
directly processes raw, unstructured photographs, which may vary significantly
in pose, viewpoint, cropping, and occlusion. Instead of compressing data into
tokens for slow online text-to-3D optimization, we introduce a data rectifier
paradigm that efficiently converts unconstrained inputs into clean, orthogonal
multi-view images in a single forward pass within seconds, simplifying the 3D
reconstruction. Central to UP2You is a pose-correlated feature aggregation
module (PCFA), that selectively fuses information from multiple reference
images w.r.t. target poses, enabling better identity preservation and nearly
constant memory footprint, with more observations. We also introduce a
perceiver-based multi-reference shape predictor, removing the need for
pre-captured body templates. Extensive experiments on 4D-Dress, PuzzleIOI, and
in-the-wild captures demonstrate that UP2You consistently surpasses previous
methods in both geometric accuracy (Chamfer-15%, P2S-18% on PuzzleIOI) and
texture fidelity (PSNR-21%, LPIPS-46% on 4D-Dress). UP2You is efficient (1.5
minutes per person), and versatile (supports arbitrary pose control, and
training-free multi-garment 3D virtual try-on), making it practical for
real-world scenarios where humans are casually captured. Both models and code
will be released to facilitate future research on this underexplored task.
Project Page: https://zcai0612.github.io/UP2You
68e6e6437ae125f9582e682c
Zeyu Cai
claimed_verified
2025-10-10T09:37:50.829Z
64b855c952b7353d8c5c6a35
/avatars/e6cb373da9721c4772a195ab731703d2.svg
Zeyu Cai
user
Co2y
68e6e6437ae125f9582e682d
Ziyang Li
68e6e6437ae125f9582e682e
Xiaoben Li
68e6e6437ae125f9582e682f
Boqian Li
68e6e6437ae125f9582e6830
Zeyu Wang
68e6e6437ae125f9582e6831
Zhenyu Zhang
claimed_verified
2025-10-11T13:49:13.333Z
642e6cc9be01b88c9446fe3d
/avatars/43a8f9bf35bea54bcda547174d921aa4.svg
Zhenyu Zhang
user
JesseZhang
68e6e6437ae125f9582e6832
Yuliang Xiu
claimed_verified
2025-10-09T06:50:53.281Z
6183d0b249ef1d984699e4a3
https://cdn-avatars.huggingface.co/v1/production/uploads/6183d0b249ef1d984699e4a3/M6B9b73v8su9H8jfdcbgu.jpeg
Yuliang Xiu
user
Yuliang
UP2You: Fast Reconstruction of Yourself from Unconstrained Photo
  Collections
We present UP2You, the first tuning-free solution for reconstructing
high-fidelity 3D clothed portraits from extremely unconstrained in-the-wild 2D
photos. Unlike previous approaches that require "clean" inputs (e.g., full-body
images with minimal occlusions, or well-calibrated cross-view captures), UP2You
directly processes raw, unstructured photographs, which may vary significantly
in pose, viewpoint, cropping, and occlusion. Instead of compressing data into
tokens for slow online text-to-3D optimization, we introduce a data rectifier
paradigm that efficiently converts unconstrained inputs into clean, orthogonal
multi-view images in a single forward pass within seconds, simplifying the 3D
reconstruction. Central to UP2You is a pose-correlated feature aggregation
module (PCFA), that selectively fuses information from multiple reference
images w.r.t. target poses, enabling better identity preservation and nearly
constant memory footprint, with more observations. We also introduce a
perceiver-based multi-reference shape predictor, removing the need for
pre-captured body templates. Extensive experiments on 4D-Dress, PuzzleIOI, and
in-the-wild captures demonstrate that UP2You consistently surpasses previous
methods in both geometric accuracy (Chamfer-15%, P2S-18% on PuzzleIOI) and
texture fidelity (PSNR-21%, LPIPS-46% on 4D-Dress). UP2You is efficient (1.5
minutes per person), and versatile (supports arbitrary pose control, and
training-free multi-garment 3D virtual try-on), making it practical for
real-world scenarios where humans are casually captured. Both models and code
will be released to facilitate future research on this underexplored task.
Project Page: https://zcai0612.github.io/UP2You
UP2You reconstructs high-fidelity 3D clothed portraits from unconstrained 2D photos using a data rectifier and pose-correlated feature aggregation, achieving superior geometric and texture accuracy.
data rectifier pose-correlated feature aggregation perceiver-based multi-reference shape predictor Chamfer P2S PSNR LPIPS 3D reconstruction 3D virtual try-on