CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction
  Model
Feed-forward 3D generative models like the Large Reconstruction Model (LRM)
have demonstrated exceptional generation speed. However, the transformer-based
methods do not leverage the geometric priors of the triplane component in their
architecture, often leading to sub-optimal quality given the limited size of 3D
data and slow training. In this work, we present the Convolutional
Reconstruction Model (CRM), a high-fidelity feed-forward single image-to-3D
generative model. Recognizing the limitations posed by sparse 3D data, we
highlight the necessity of integrating geometric priors into network design.
CRM builds on the key observation that the visualization of triplane exhibits
spatial correspondence of six orthographic images. First, it generates six
orthographic view images from a single input image, then feeds these images
into a convolutional U-Net, leveraging its strong pixel-level alignment
capabilities and significant bandwidth to create a high-resolution triplane.
CRM further employs Flexicubes as geometric representation, facilitating direct
end-to-end optimization on textured meshes. Overall, our model delivers a
high-fidelity textured mesh from an image in just 10 seconds, without any
test-time optimization.
65ee693554082af7dbd87ee1
Zhengyi Wang
claimed_verified
2024-03-11T08:20:17.585Z
634e15aec1ce28f1de91c470
https://cdn-avatars.huggingface.co/v1/production/uploads/634e15aec1ce28f1de91c470/MdtJKgE_fbHydhqESr-DH.png
Zhengyi Wang
user
Zhengyi
65ee693554082af7dbd87ee2
Yikai Wang
claimed_verified
2024-03-13T13:26:46.629Z
64206b35e40f66bcd1e3c985
/avatars/a8f830d1c47702f642045399e59c9fe8.svg
Yikai Wang
user
yikaiw
65ee693554082af7dbd87ee3
Yifei Chen
65ee693554082af7dbd87ee4
Chendong Xiang
admin_assigned
2024-03-11T09:05:42.923Z
6329bdbbde087eac2921e6a9
https://cdn-avatars.huggingface.co/v1/production/uploads/1663679904323-noauth.jpeg
Xiangchendong
user
Xiang-cd
65ee693554082af7dbd87ee5
Shuo Chen
claimed_verified
2024-03-12T12:06:26.596Z
6556031ab7bad186e86e96e7
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/_SjKb8h1N8URnQchqu327.jpeg
ShuoChen
user
ShuoChen20
65ee693554082af7dbd87ee6
Dajiang Yu
65ee693554082af7dbd87ee7
Chongxuan Li
admin_assigned
2024-03-11T09:06:13.667Z
64c07b488e2612254361153b
/avatars/ade0f783cc4c2d3e73f402637f595471.svg
chongxuan li
user
zhenxuan00
65ee693554082af7dbd87ee8
Hang Su
65ee693554082af7dbd87ee9
Jun Zhu
CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction
  Model
Feed-forward 3D generative models like the Large Reconstruction Model (LRM)
have demonstrated exceptional generation speed. However, the transformer-based
methods do not leverage the geometric priors of the triplane component in their
architecture, often leading to sub-optimal quality given the limited size of 3D
data and slow training. In this work, we present the Convolutional
Reconstruction Model (CRM), a high-fidelity feed-forward single image-to-3D
generative model. Recognizing the limitations posed by sparse 3D data, we
highlight the necessity of integrating geometric priors into network design.
CRM builds on the key observation that the visualization of triplane exhibits
spatial correspondence of six orthographic images. First, it generates six
orthographic view images from a single input image, then feeds these images
into a convolutional U-Net, leveraging its strong pixel-level alignment
capabilities and significant bandwidth to create a high-resolution triplane.
CRM further employs Flexicubes as geometric representation, facilitating direct
end-to-end optimization on textured meshes. Overall, our model delivers a
high-fidelity textured mesh from an image in just 10 seconds, without any
test-time optimization.
A convolutional generative model that integrates geometric priors and leverages Flexicubes for efficient high-fidelity 3D mesh generation from a single image.
feed-forward 3D generative models Large Reconstruction Model (LRM) transformer-based methods geometric priors triplane component Convolutional Reconstruction Model (CRM) convolutional U-Net pixel-level alignment textured meshes Flexicubes