Customize your NeRF: Adaptive Source Driven 3D Scene Editing via
  Local-Global Iterative Training
In this paper, we target the adaptive source driven 3D scene editing task by
proposing a CustomNeRF model that unifies a text description or a reference
image as the editing prompt. However, obtaining desired editing results
conformed with the editing prompt is nontrivial since there exist two
significant challenges, including accurate editing of only foreground regions
and multi-view consistency given a single-view reference image. To tackle the
first challenge, we propose a Local-Global Iterative Editing (LGIE) training
scheme that alternates between foreground region editing and full-image
editing, aimed at foreground-only manipulation while preserving the background.
For the second challenge, we also design a class-guided regularization that
exploits class priors within the generation model to alleviate the
inconsistency problem among different views in image-driven editing. Extensive
experiments show that our CustomNeRF produces precise editing results under
various real scenes for both text- and image-driven settings.
656e9f847e3e612b355b9b51
Runze He
656e9f847e3e612b355b9b52
Shaofei Huang
656e9f847e3e612b355b9b53
Xuecheng Nie
656e9f847e3e612b355b9b54
Tianrui Hui
656e9f847e3e612b355b9b55
Luoqi Liu
656e9f847e3e612b355b9b56
Jiao Dai
656e9f847e3e612b355b9b57
Jizhong Han
656e9f847e3e612b355b9b58
Guanbin Li
656e9f847e3e612b355b9b59
Si Liu
Customize your NeRF: Adaptive Source Driven 3D Scene Editing via
  Local-Global Iterative Training
In this paper, we target the adaptive source driven 3D scene editing task by
proposing a CustomNeRF model that unifies a text description or a reference
image as the editing prompt. However, obtaining desired editing results
conformed with the editing prompt is nontrivial since there exist two
significant challenges, including accurate editing of only foreground regions
and multi-view consistency given a single-view reference image. To tackle the
first challenge, we propose a Local-Global Iterative Editing (LGIE) training
scheme that alternates between foreground region editing and full-image
editing, aimed at foreground-only manipulation while preserving the background.
For the second challenge, we also design a class-guided regularization that
exploits class priors within the generation model to alleviate the
inconsistency problem among different views in image-driven editing. Extensive
experiments show that our CustomNeRF produces precise editing results under
various real scenes for both text- and image-driven settings.
A CustomNeRF model addresses challenges in 3D scene editing by using text or image prompts, with a Local-Global Iterative Editing scheme and class-guided regularization to improve accuracy and consistency.
CustomNeRF Local-Global Iterative Editing (LGIE) class-guided regularization