FlexiDreamer: Single Image-to-3D Generation with FlexiCubes
3D content generation from text prompts or single images has made remarkable
progress in quality and speed recently. One of its dominant paradigms involves
generating consistent multi-view images followed by a sparse-view
reconstruction. However, due to the challenge of directly deforming the mesh
representation to approach the target topology, most methodologies learn an
implicit representation (such as NeRF) during the sparse-view reconstruction
and acquire the target mesh by a post-processing extraction. Although the
implicit representation can effectively model rich 3D information, its training
typically entails a long convergence time. In addition, the post-extraction
operation from the implicit field also leads to undesirable visual artifacts.
In this paper, we propose FlexiDreamer, a novel single image-to-3d generation
framework that reconstructs the target mesh in an end-to-end manner. By
leveraging a flexible gradient-based extraction known as FlexiCubes, our method
circumvents the defects brought by the post-processing and facilitates a direct
acquisition of the target mesh. Furthermore, we incorporate a multi-resolution
hash grid encoding scheme that progressively activates the encoding levels into
the implicit field in FlexiCubes to help capture geometric details for per-step
optimization. Notably, FlexiDreamer recovers a dense 3D structure from a
single-view image in approximately 1 minute on a single NVIDIA A100 GPU,
outperforming previous methodologies by a large margin.
660b96033fbd33a1d0f703dc
Ruowen Zhao
claimed_verified
2024-04-02T11:02:58.599Z
6522e4fbd89bc7773ddc4b58
/avatars/3e9b158af52c5f738a3eae72dcbb3824.svg
Ruowen Zhao
user
zzzrw
660b96033fbd33a1d0f703dd
Zhengyi Wang
claimed_verified
2024-04-03T09:27:14.382Z
634e15aec1ce28f1de91c470
https://cdn-avatars.huggingface.co/v1/production/uploads/634e15aec1ce28f1de91c470/MdtJKgE_fbHydhqESr-DH.png
Zhengyi Wang
user
Zhengyi
660b96033fbd33a1d0f703de
Yikai Wang
660b96033fbd33a1d0f703df
Zihan Zhou
660b96033fbd33a1d0f703e0
Jun Zhu
FlexiDreamer: Single Image-to-3D Generation with FlexiCubes
3D content generation from text prompts or single images has made remarkable
progress in quality and speed recently. One of its dominant paradigms involves
generating consistent multi-view images followed by a sparse-view
reconstruction. However, due to the challenge of directly deforming the mesh
representation to approach the target topology, most methodologies learn an
implicit representation (such as NeRF) during the sparse-view reconstruction
and acquire the target mesh by a post-processing extraction. Although the
implicit representation can effectively model rich 3D information, its training
typically entails a long convergence time. In addition, the post-extraction
operation from the implicit field also leads to undesirable visual artifacts.
In this paper, we propose FlexiDreamer, a novel single image-to-3d generation
framework that reconstructs the target mesh in an end-to-end manner. By
leveraging a flexible gradient-based extraction known as FlexiCubes, our method
circumvents the defects brought by the post-processing and facilitates a direct
acquisition of the target mesh. Furthermore, we incorporate a multi-resolution
hash grid encoding scheme that progressively activates the encoding levels into
the implicit field in FlexiCubes to help capture geometric details for per-step
optimization. Notably, FlexiDreamer recovers a dense 3D structure from a
single-view image in approximately 1 minute on a single NVIDIA A100 GPU,
outperforming previous methodologies by a large margin.
FlexiDreamer is a novel end-to-end framework for single-image 3D generation using FlexiCubes and a multi-resolution hash grid encoding scheme to improve efficiency and reduce visual artifacts.
NeRF implicit representation sparse-view reconstruction gradient-based extraction FlexiCubes multi-resolution hash grid encoding end-to-end generation geometric details per-step optimization dense 3D structure 3D content generation single-image-to-3D generation