Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side
  Images
Recent advances in 3D AIGC have shown promise in directly creating 3D objects
from text and images, offering significant cost savings in animation and
product design. However, detailed edit and customization of 3D assets remains a
long-standing challenge. Specifically, 3D Generation methods lack the ability
to follow finely detailed instructions as precisely as their 2D image creation
counterparts. Imagine you can get a toy through 3D AIGC but with undesired
accessories and dressing. To tackle this challenge, we propose a novel pipeline
called Tailor3D, which swiftly creates customized 3D assets from editable
dual-side images. We aim to emulate a tailor's ability to locally change
objects or perform overall style transfer. Unlike creating 3D assets from
multiple views, using dual-side images eliminates conflicts on overlapping
areas that occur when editing individual views. Specifically, it begins by
editing the front view, then generates the back view of the object through
multi-view diffusion. Afterward, it proceeds to edit the back views. Finally, a
Dual-sided LRM is proposed to seamlessly stitch together the front and back 3D
features, akin to a tailor sewing together the front and back of a garment. The
Dual-sided LRM rectifies imperfect consistencies between the front and back
views, enhancing editing capabilities and reducing memory burdens while
seamlessly integrating them into a unified 3D representation with the LoRA
Triplane Transformer. Experimental results demonstrate Tailor3D's effectiveness
across various 3D generation and editing tasks, including 3D generative fill
and style transfer. It provides a user-friendly, efficient solution for editing
3D assets, with each editing step taking only seconds to complete.
668cb6cc61b6eff5a82aeba2
Zhangyang Qi
admin_assigned
2024-07-09T14:05:53.230Z
64c129c155686a357c99ea0b
https://cdn-avatars.huggingface.co/v1/production/uploads/64c129c155686a357c99ea0b/1iy8_aLVU4FflO-d1fozO.jpeg
Qi Zhangyang (Alex Severns)
user
alexzyqi
668cb6cc61b6eff5a82aeba3
Yunhan Yang
admin_assigned
2024-07-09T14:06:02.450Z
64b4eecf2fc8324fcb63b404
https://cdn-avatars.huggingface.co/v1/production/uploads/64b4eecf2fc8324fcb63b404/zGYqYVB4-o-GBMybJ8CDA.png
Yunhan Yang
user
yhyang-myron
668cb6cc61b6eff5a82aeba4
Mengchen Zhang
admin_assigned
2024-07-09T14:06:16.585Z
64de20c5808492ba6e65d124
https://cdn-avatars.huggingface.co/v1/production/uploads/64de20c5808492ba6e65d124/58IX_TI5vJw73qS1knw56.jpeg
Zhang Mengchen
user
Dubhe-zmc
668cb6cc61b6eff5a82aeba5
Long Xing
668cb6cc61b6eff5a82aeba6
Xiaoyang Wu
admin_assigned
2024-07-09T14:09:24.638Z
643e5d6a1d0e956d94bb3608
https://cdn-avatars.huggingface.co/v1/production/uploads/643e5d6a1d0e956d94bb3608/mTnu7Dts2RmDklHlp9Gqu.jpeg
Xiaoyang Wu
user
Gofinge
668cb6cc61b6eff5a82aeba7
Tong Wu
claimed_verified
2024-12-11T09:23:19.248Z
6310929fda0aaad2f70397a1
https://cdn-avatars.huggingface.co/v1/production/uploads/6310929fda0aaad2f70397a1/YoHeYZezmCOXwL-qsiZOc.png
Tong Wu
user
wutong16
668cb6cc61b6eff5a82aeba8
Dahua Lin
admin_assigned
2024-07-09T14:08:43.006Z
636317ed80c1a705a6eff396
/avatars/3db090e101b916d9256d0d3e043db71d.svg
Dahua Lin
user
lindahua
668cb6cc61b6eff5a82aeba9
Xihui Liu
claimed_verified
2024-10-16T09:44:52.854Z
65d5ec74cd05bc1eaa125040
/avatars/2de1b1539a86452c2c89570eeb02f5ab.svg
Xihui Liu
user
XihuiLiu
668cb6cc61b6eff5a82aebaa
Jiaqi Wang
admin_assigned
2024-07-09T14:07:59.442Z
64638c4d51fa6e63060521b5
/avatars/c863ace5b1dc788a341bcf4ddbdfaec1.svg
JIaqi
user
Jiaqiwang
668cb6cc61b6eff5a82aebab
Hengshuang Zhao
Tailor3D: Customized 3D Assets Editing and Generation with Dual-Side
  Images
Recent advances in 3D AIGC have shown promise in directly creating 3D objects
from text and images, offering significant cost savings in animation and
product design. However, detailed edit and customization of 3D assets remains a
long-standing challenge. Specifically, 3D Generation methods lack the ability
to follow finely detailed instructions as precisely as their 2D image creation
counterparts. Imagine you can get a toy through 3D AIGC but with undesired
accessories and dressing. To tackle this challenge, we propose a novel pipeline
called Tailor3D, which swiftly creates customized 3D assets from editable
dual-side images. We aim to emulate a tailor's ability to locally change
objects or perform overall style transfer. Unlike creating 3D assets from
multiple views, using dual-side images eliminates conflicts on overlapping
areas that occur when editing individual views. Specifically, it begins by
editing the front view, then generates the back view of the object through
multi-view diffusion. Afterward, it proceeds to edit the back views. Finally, a
Dual-sided LRM is proposed to seamlessly stitch together the front and back 3D
features, akin to a tailor sewing together the front and back of a garment. The
Dual-sided LRM rectifies imperfect consistencies between the front and back
views, enhancing editing capabilities and reducing memory burdens while
seamlessly integrating them into a unified 3D representation with the LoRA
Triplane Transformer. Experimental results demonstrate Tailor3D's effectiveness
across various 3D generation and editing tasks, including 3D generative fill
and style transfer. It provides a user-friendly, efficient solution for editing
3D assets, with each editing step taking only seconds to complete.
Tailor3D addresses 3D asset customization by using dual-side images and a novel Dual-sided LRM for seamless integration, enhancing editing efficiency and consistency.
3D Generation dual-side images multi-view diffusion Dual-sided LRM LoRA Triplane Transformer 3D generative fill style transfer