Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields
With the popularity of implicit neural representations, or neural radiance
fields (NeRF), there is a pressing need for editing methods to interact with
the implicit 3D models for tasks like post-processing reconstructed scenes and
3D content creation. While previous works have explored NeRF editing from
various perspectives, they are restricted in editing flexibility, quality, and
speed, failing to offer direct editing response and instant preview. The key
challenge is to conceive a locally editable neural representation that can
directly reflect the editing instructions and update instantly. To bridge the
gap, we propose a new interactive editing method and system for implicit
representations, called Seal-3D, which allows users to edit NeRF models in a
pixel-level and free manner with a wide range of NeRF-like backbone and preview
the editing effects instantly. To achieve the effects, the challenges are
addressed by our proposed proxy function mapping the editing instructions to
the original space of NeRF models and a teacher-student training strategy with
local pretraining and global finetuning. A NeRF editing system is built to
showcase various editing types. Our system can achieve compelling editing
effects with an interactive speed of about 1 second.
64c74febaedc4337783d06e0
Xiangyu Wang
64c74febaedc4337783d06e1
Jingsen Zhu
64c74febaedc4337783d06e2
Qi Ye
64c74febaedc4337783d06e3
Yuchi Huo
64c74febaedc4337783d06e4
Yunlong Ran
64c74febaedc4337783d06e5
Zhihua Zhong
64c74febaedc4337783d06e6
Jiming Chen
Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields
With the popularity of implicit neural representations, or neural radiance
fields (NeRF), there is a pressing need for editing methods to interact with
the implicit 3D models for tasks like post-processing reconstructed scenes and
3D content creation. While previous works have explored NeRF editing from
various perspectives, they are restricted in editing flexibility, quality, and
speed, failing to offer direct editing response and instant preview. The key
challenge is to conceive a locally editable neural representation that can
directly reflect the editing instructions and update instantly. To bridge the
gap, we propose a new interactive editing method and system for implicit
representations, called Seal-3D, which allows users to edit NeRF models in a
pixel-level and free manner with a wide range of NeRF-like backbone and preview
the editing effects instantly. To achieve the effects, the challenges are
addressed by our proposed proxy function mapping the editing instructions to
the original space of NeRF models and a teacher-student training strategy with
local pretraining and global finetuning. A NeRF editing system is built to
showcase various editing types. Our system can achieve compelling editing
effects with an interactive speed of about 1 second.
Seal-3D enables interactive and free-form pixel-level editing of NeRF models with instant preview using a proxy function and teacher-student training strategy.
implicit neural representations neural radiance fields NeRF local editing interactive editing proxy function teacher-student training local pretraining global finetuning