SHAP-EDITOR: Instruction-guided Latent 3D Editing in Seconds
We propose a novel feed-forward 3D editing framework called Shap-Editor.
Prior research on editing 3D objects primarily concentrated on editing
individual objects by leveraging off-the-shelf 2D image editing networks. This
is achieved via a process called distillation, which transfers knowledge from
the 2D network to 3D assets. Distillation necessitates at least tens of minutes
per asset to attain satisfactory editing results, and is thus not very
practical. In contrast, we ask whether 3D editing can be carried out directly
by a feed-forward network, eschewing test-time optimisation. In particular, we
hypothesise that editing can be greatly simplified by first encoding 3D objects
in a suitable latent space. We validate this hypothesis by building upon the
latent space of Shap-E. We demonstrate that direct 3D editing in this space is
possible and efficient by building a feed-forward editor network that only
requires approximately one second per edit. Our experiments show that
Shap-Editor generalises well to both in-distribution and out-of-distribution 3D
assets with different prompts, exhibiting comparable performance with methods
that carry out test-time optimisation for each edited instance.
657bc2aae34a7de14b0b3b1f
Minghao Chen
claimed_verified
2023-12-15T16:14:55.226Z
636a3d8bf8d9af4aea18553f
/avatars/028a86d088764fd66c36a2ddebf09f9a.svg
MINGHAO CHEN
user
silentchen
657bc2aae34a7de14b0b3b20
Junyu Xie
657bc2aae34a7de14b0b3b21
Iro Laina
657bc2aae34a7de14b0b3b22
Andrea Vedaldi
SHAP-EDITOR: Instruction-guided Latent 3D Editing in Seconds
We propose a novel feed-forward 3D editing framework called Shap-Editor.
Prior research on editing 3D objects primarily concentrated on editing
individual objects by leveraging off-the-shelf 2D image editing networks. This
is achieved via a process called distillation, which transfers knowledge from
the 2D network to 3D assets. Distillation necessitates at least tens of minutes
per asset to attain satisfactory editing results, and is thus not very
practical. In contrast, we ask whether 3D editing can be carried out directly
by a feed-forward network, eschewing test-time optimisation. In particular, we
hypothesise that editing can be greatly simplified by first encoding 3D objects
in a suitable latent space. We validate this hypothesis by building upon the
latent space of Shap-E. We demonstrate that direct 3D editing in this space is
possible and efficient by building a feed-forward editor network that only
requires approximately one second per edit. Our experiments show that
Shap-Editor generalises well to both in-distribution and out-of-distribution 3D
assets with different prompts, exhibiting comparable performance with methods
that carry out test-time optimisation for each edited instance.
A feed-forward network named Shap-Editor performs efficient 3D editing in a latent space derived from Shap-E, achieving fast and high-quality editing results.
feed-forward 3D editing latent space distillation test-time optimisation Shap-E in-distribution out-of-distribution