PartCrafter: Structured 3D Mesh Generation via Compositional Latent
  Diffusion Transformers
We introduce PartCrafter, the first structured 3D generative model that
jointly synthesizes multiple semantically meaningful and geometrically distinct
3D meshes from a single RGB image. Unlike existing methods that either produce
monolithic 3D shapes or follow two-stage pipelines, i.e., first segmenting an
image and then reconstructing each segment, PartCrafter adopts a unified,
compositional generation architecture that does not rely on pre-segmented
inputs. Conditioned on a single image, it simultaneously denoises multiple 3D
parts, enabling end-to-end part-aware generation of both individual objects and
complex multi-object scenes. PartCrafter builds upon a pretrained 3D mesh
diffusion transformer (DiT) trained on whole objects, inheriting the pretrained
weights, encoder, and decoder, and introduces two key innovations: (1) A
compositional latent space, where each 3D part is represented by a set of
disentangled latent tokens; (2) A hierarchical attention mechanism that enables
structured information flow both within individual parts and across all parts,
ensuring global coherence while preserving part-level detail during generation.
To support part-level supervision, we curate a new dataset by mining part-level
annotations from large-scale 3D object datasets. Experiments show that
PartCrafter outperforms existing approaches in generating decomposable 3D
meshes, including parts that are not directly visible in input images,
demonstrating the strength of part-aware generative priors for 3D understanding
and synthesis. Code and training data will be released.
6846902a3ec10bdd8ab4db61
Yuchen Lin
claimed_verified
2025-06-10T09:21:53.647Z
67b2b698ba726eda5cbd07da
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/qZvmHqsxV2lXfYz2Wx3rA.jpeg
Yuchen Lin
user
wgsxm
6846902a3ec10bdd8ab4db62
Chenguo Lin
claimed_verified
2025-06-09T10:11:09.640Z
62e18206926f4892a4c782bd
https://cdn-avatars.huggingface.co/v1/production/uploads/62e18206926f4892a4c782bd/m_OAeRb2a7a5PlnK6d4Dg.jpeg
Chenguo Lin
user
chenguolin
6846902a3ec10bdd8ab4db63
Panwang Pan
claimed_verified
2025-06-10T09:21:55.707Z
654866e8cd0a5621395f8287
https://cdn-avatars.huggingface.co/v1/production/uploads/654866e8cd0a5621395f8287/4Bccwd1ehn-Ee4T1rId5S.jpeg
Panwang Pan
user
paulpanwang
6846902a3ec10bdd8ab4db64
Honglei Yan
6846902a3ec10bdd8ab4db65
Yiqiang Feng
6846902a3ec10bdd8ab4db66
Yadong Mu
6846902a3ec10bdd8ab4db67
Katerina Fragkiadaki
PartCrafter: Structured 3D Mesh Generation via Compositional Latent
  Diffusion Transformers
We introduce PartCrafter, the first structured 3D generative model that
jointly synthesizes multiple semantically meaningful and geometrically distinct
3D meshes from a single RGB image. Unlike existing methods that either produce
monolithic 3D shapes or follow two-stage pipelines, i.e., first segmenting an
image and then reconstructing each segment, PartCrafter adopts a unified,
compositional generation architecture that does not rely on pre-segmented
inputs. Conditioned on a single image, it simultaneously denoises multiple 3D
parts, enabling end-to-end part-aware generation of both individual objects and
complex multi-object scenes. PartCrafter builds upon a pretrained 3D mesh
diffusion transformer (DiT) trained on whole objects, inheriting the pretrained
weights, encoder, and decoder, and introduces two key innovations: (1) A
compositional latent space, where each 3D part is represented by a set of
disentangled latent tokens; (2) A hierarchical attention mechanism that enables
structured information flow both within individual parts and across all parts,
ensuring global coherence while preserving part-level detail during generation.
To support part-level supervision, we curate a new dataset by mining part-level
annotations from large-scale 3D object datasets. Experiments show that
PartCrafter outperforms existing approaches in generating decomposable 3D
meshes, including parts that are not directly visible in input images,
demonstrating the strength of part-aware generative priors for 3D understanding
and synthesis. Code and training data will be released.
PartCrafter is a unified 3D generative model that synthesizes multiple semantically meaningful 3D meshes from a single image using a compositional latent space and hierarchical attention mechanism.
3D generative model multiple 3D meshes RGB image unified compositional generation architecture denoising 3D diffusion transformer (DiT) compositional latent space disentangled latent tokens hierarchical attention mechanism part-level supervision part-aware generative priors