3D-Fixup: Advancing Photo Editing with 3D Priors
Despite significant advances in modeling image priors via diffusion models,
3D-aware image editing remains challenging, in part because the object is only
specified via a single image. To tackle this challenge, we propose 3D-Fixup, a
new framework for editing 2D images guided by learned 3D priors. The framework
supports difficult editing situations such as object translation and 3D
rotation. To achieve this, we leverage a training-based approach that harnesses
the generative power of diffusion models. As video data naturally encodes
real-world physical dynamics, we turn to video data for generating training
data pairs, i.e., a source and a target frame. Rather than relying solely on a
single trained model to infer transformations between source and target frames,
we incorporate 3D guidance from an Image-to-3D model, which bridges this
challenging task by explicitly projecting 2D information into 3D space. We
design a data generation pipeline to ensure high-quality 3D guidance throughout
training. Results show that by integrating these 3D priors, 3D-Fixup
effectively supports complex, identity coherent 3D-aware edits, achieving
high-quality results and advancing the application of diffusion models in
realistic image manipulation. The code is provided at
https://3dfixup.github.io/
6826a515cf1804ab4c37cde8
Yen-Chi Cheng
extracted_confirmed
2025-05-16T18:15:41.266Z
634cbe8888c6b621d3577fe6
/avatars/e8108ccb3602a5e387a03aaa06d7fdc3.svg
Yen-Chi Cheng
user
yenchic
6826a515cf1804ab4c37cde9
Krishna Kumar Singh
6826a515cf1804ab4c37cdea
Jae Shin Yoon
6826a515cf1804ab4c37cdeb
Alex Schwing
6826a515cf1804ab4c37cdec
Liangyan Gui
6826a515cf1804ab4c37cded
Matheus Gadelha
6826a515cf1804ab4c37cdee
Paul Guerrero
6826a515cf1804ab4c37cdef
Nanxuan Zhao
3D-Fixup: Advancing Photo Editing with 3D Priors
Despite significant advances in modeling image priors via diffusion models,
3D-aware image editing remains challenging, in part because the object is only
specified via a single image. To tackle this challenge, we propose 3D-Fixup, a
new framework for editing 2D images guided by learned 3D priors. The framework
supports difficult editing situations such as object translation and 3D
rotation. To achieve this, we leverage a training-based approach that harnesses
the generative power of diffusion models. As video data naturally encodes
real-world physical dynamics, we turn to video data for generating training
data pairs, i.e., a source and a target frame. Rather than relying solely on a
single trained model to infer transformations between source and target frames,
we incorporate 3D guidance from an Image-to-3D model, which bridges this
challenging task by explicitly projecting 2D information into 3D space. We
design a data generation pipeline to ensure high-quality 3D guidance throughout
training. Results show that by integrating these 3D priors, 3D-Fixup
effectively supports complex, identity coherent 3D-aware edits, achieving
high-quality results and advancing the application of diffusion models in
realistic image manipulation. The code is provided at
https://3dfixup.github.io/
A new framework, 3D-Fixup, improves 3D-aware image editing by integrating 3D priors learned from video data into diffusion models.
3D-Fixup diffusion models 3D priors Image-to-3D model 3D guidance data generation pipeline 3D-aware edits realistic image manipulation