ObjFiller-3D: Consistent Multi-view 3D Inpainting via Video Diffusion
  Models
3D inpainting often relies on multi-view 2D image inpainting, where the
inherent inconsistencies across different inpainted views can result in blurred
textures, spatial discontinuities, and distracting visual artifacts. These
inconsistencies pose significant challenges when striving for accurate and
realistic 3D object completion, particularly in applications that demand high
fidelity and structural coherence. To overcome these limitations, we propose
ObjFiller-3D, a novel method designed for the completion and editing of
high-quality and consistent 3D objects. Instead of employing a conventional 2D
image inpainting model, our approach leverages a curated selection of
state-of-the-art video editing model to fill in the masked regions of 3D
objects. We analyze the representation gap between 3D and videos, and propose
an adaptation of a video inpainting model for 3D scene inpainting. In addition,
we introduce a reference-based 3D inpainting method to further enhance the
quality of reconstruction. Experiments across diverse datasets show that
compared to previous methods, ObjFiller-3D produces more faithful and
fine-grained reconstructions (PSNR of 26.6 vs. NeRFiller (15.9) and LPIPS of
0.19 vs. Instant3dit (0.25)). Moreover, it demonstrates strong potential for
practical deployment in real-world 3D editing applications. Project page:
https://objfiller3d.github.io/ Code:
https://github.com/objfiller3d/ObjFiller-3D .
68ae8b74364411bea07df828
Haitang Feng
68ae8b74364411bea07df829
Jie Liu
68ae8b74364411bea07df82a
Jie Tang
68ae8b74364411bea07df82b
Gangshan Wu
68ae8b74364411bea07df82c
Beiqi Chen
68ae8b74364411bea07df82d
Jianhuang Lai
68ae8b74364411bea07df82e
Guangcong Wang
ObjFiller-3D: Consistent Multi-view 3D Inpainting via Video Diffusion
  Models
3D inpainting often relies on multi-view 2D image inpainting, where the
inherent inconsistencies across different inpainted views can result in blurred
textures, spatial discontinuities, and distracting visual artifacts. These
inconsistencies pose significant challenges when striving for accurate and
realistic 3D object completion, particularly in applications that demand high
fidelity and structural coherence. To overcome these limitations, we propose
ObjFiller-3D, a novel method designed for the completion and editing of
high-quality and consistent 3D objects. Instead of employing a conventional 2D
image inpainting model, our approach leverages a curated selection of
state-of-the-art video editing model to fill in the masked regions of 3D
objects. We analyze the representation gap between 3D and videos, and propose
an adaptation of a video inpainting model for 3D scene inpainting. In addition,
we introduce a reference-based 3D inpainting method to further enhance the
quality of reconstruction. Experiments across diverse datasets show that
compared to previous methods, ObjFiller-3D produces more faithful and
fine-grained reconstructions (PSNR of 26.6 vs. NeRFiller (15.9) and LPIPS of
0.19 vs. Instant3dit (0.25)). Moreover, it demonstrates strong potential for
practical deployment in real-world 3D editing applications. Project page:
https://objfiller3d.github.io/ Code:
https://github.com/objfiller3d/ObjFiller-3D .
ObjFiller-3D uses video editing models to achieve high-quality and consistent 3D object completion, outperforming previous methods in terms of reconstruction fidelity and practical deployment.
3D inpainting multi-view 2D image inpainting video editing model 3D scene inpainting reference-based 3D inpainting PSNR LPIPS