MeshLLM: Empowering Large Language Models to Progressively Understand
  and Generate 3D Mesh
We present MeshLLM, a novel framework that leverages large language models
(LLMs) to understand and generate text-serialized 3D meshes. Our approach
addresses key limitations in existing methods, including the limited dataset
scale when catering to LLMs' token length and the loss of 3D structural
information during mesh serialization. We introduce a Primitive-Mesh
decomposition strategy, which divides 3D meshes into structurally meaningful
subunits. This enables the creation of a large-scale dataset with 1500k+
samples, almost 50 times larger than previous methods, which aligns better with
the LLM scaling law principles. Furthermore, we propose inferring face
connectivity from vertices and local mesh assembly training strategies,
significantly enhancing the LLMs' ability to capture mesh topology and spatial
structures. Experiments show that MeshLLM outperforms the state-of-the-art
LLaMA-Mesh in both mesh generation quality and shape understanding,
highlighting its great potential in processing text-serialized 3D meshes.
68997c1bf022d141f5d43616
Shuangkang Fang
68997c1bf022d141f5d43617
I-Chao Shen
68997c1bf022d141f5d43618
Yufeng Wang
68997c1bf022d141f5d43619
Yi-Hsuan Tsai
68997c1bf022d141f5d4361a
Yi Yang
68997c1bf022d141f5d4361b
Shuchang Zhou
68997c1bf022d141f5d4361c
Wenrui Ding
68997c1bf022d141f5d4361d
Takeo Igarashi
68997c1bf022d141f5d4361e
Ming-Hsuan Yang
MeshLLM: Empowering Large Language Models to Progressively Understand
  and Generate 3D Mesh
We present MeshLLM, a novel framework that leverages large language models
(LLMs) to understand and generate text-serialized 3D meshes. Our approach
addresses key limitations in existing methods, including the limited dataset
scale when catering to LLMs' token length and the loss of 3D structural
information during mesh serialization. We introduce a Primitive-Mesh
decomposition strategy, which divides 3D meshes into structurally meaningful
subunits. This enables the creation of a large-scale dataset with 1500k+
samples, almost 50 times larger than previous methods, which aligns better with
the LLM scaling law principles. Furthermore, we propose inferring face
connectivity from vertices and local mesh assembly training strategies,
significantly enhancing the LLMs' ability to capture mesh topology and spatial
structures. Experiments show that MeshLLM outperforms the state-of-the-art
LLaMA-Mesh in both mesh generation quality and shape understanding,
highlighting its great potential in processing text-serialized 3D meshes.
MeshLLM uses large language models to generate and understand text-serialized 3D meshes by decomposing them into meaningful subunits and training with local mesh assembly strategies.
MeshLLM large language models LLMs 3D meshes Primitive-Mesh decomposition face connectivity local mesh assembly mesh topology spatial structures LLaMA-Mesh