Pandora3D: A Comprehensive Framework for High-Quality 3D Shape and
  Texture Generation
This report presents a comprehensive framework for generating high-quality 3D
shapes and textures from diverse input prompts, including single images,
multi-view images, and text descriptions. The framework consists of 3D shape
generation and texture generation. (1). The 3D shape generation pipeline
employs a Variational Autoencoder (VAE) to encode implicit 3D geometries into a
latent space and a diffusion network to generate latents conditioned on input
prompts, with modifications to enhance model capacity. An alternative
Artist-Created Mesh (AM) generation approach is also explored, yielding
promising results for simpler geometries. (2). Texture generation involves a
multi-stage process starting with frontal images generation followed by
multi-view images generation, RGB-to-PBR texture conversion, and
high-resolution multi-view texture refinement. A consistency scheduler is
plugged into every stage, to enforce pixel-wise consistency among multi-view
textures during inference, ensuring seamless integration.
  The pipeline demonstrates effective handling of diverse input formats,
leveraging advanced neural architectures and novel methodologies to produce
high-quality 3D content. This report details the system architecture,
experimental results, and potential future directions to improve and expand the
framework. The source code and pretrained weights are released at:
https://github.com/Tencent/Tencent-XR-3DGen.
67be3ebc1c80786468704721
Jiayu Yang
67be3ebc1c80786468704722
Taizhang Shang
67be3ebc1c80786468704723
Weixuan Sun
67be3ebc1c80786468704724
Xibin Song
67be3ebc1c80786468704725
Ziang Cheng
67be3ebc1c80786468704726
Senbo Wang
67be3ebc1c80786468704727
Shenzhou Chen
67be3ebc1c80786468704728
Weizhe Liu
67be3ebc1c80786468704729
Hongdong Li
67be3ebc1c8078646870472a
Pan Ji
Pandora3D: A Comprehensive Framework for High-Quality 3D Shape and
  Texture Generation
This report presents a comprehensive framework for generating high-quality 3D
shapes and textures from diverse input prompts, including single images,
multi-view images, and text descriptions. The framework consists of 3D shape
generation and texture generation. (1). The 3D shape generation pipeline
employs a Variational Autoencoder (VAE) to encode implicit 3D geometries into a
latent space and a diffusion network to generate latents conditioned on input
prompts, with modifications to enhance model capacity. An alternative
Artist-Created Mesh (AM) generation approach is also explored, yielding
promising results for simpler geometries. (2). Texture generation involves a
multi-stage process starting with frontal images generation followed by
multi-view images generation, RGB-to-PBR texture conversion, and
high-resolution multi-view texture refinement. A consistency scheduler is
plugged into every stage, to enforce pixel-wise consistency among multi-view
textures during inference, ensuring seamless integration.
  The pipeline demonstrates effective handling of diverse input formats,
leveraging advanced neural architectures and novel methodologies to produce
high-quality 3D content. This report details the system architecture,
experimental results, and potential future directions to improve and expand the
framework. The source code and pretrained weights are released at:
https://github.com/Tencent/Tencent-XR-3DGen.
The framework generates high-quality 3D shapes and textures from various inputs using VAE, diffusion networks, and a multi-stage texture generation process with consistency enforcement.
Variational Autoencoder VAE diffusion network Artist-Created Mesh AM RGB-to-PBR consistency scheduler