3D-GPT: Procedural 3D Modeling with Large Language Models
In the pursuit of efficient automated content creation, procedural
generation, leveraging modifiable parameters and rule-based systems, emerges as
a promising approach. Nonetheless, it could be a demanding endeavor, given its
intricate nature necessitating a deep understanding of rules, algorithms, and
parameters. To reduce workload, we introduce 3D-GPT, a framework utilizing
large language models~(LLMs) for instruction-driven 3D modeling. 3D-GPT
positions LLMs as proficient problem solvers, dissecting the procedural 3D
modeling tasks into accessible segments and appointing the apt agent for each
task. 3D-GPT integrates three core agents: the task dispatch agent, the
conceptualization agent, and the modeling agent. They collaboratively achieve
two objectives. First, it enhances concise initial scene descriptions, evolving
them into detailed forms while dynamically adapting the text based on
subsequent instructions. Second, it integrates procedural generation,
extracting parameter values from enriched text to effortlessly interface with
3D software for asset creation. Our empirical investigations confirm that
3D-GPT not only interprets and executes instructions, delivering reliable
results but also collaborates effectively with human designers. Furthermore, it
seamlessly integrates with Blender, unlocking expanded manipulation
possibilities. Our work highlights the potential of LLMs in 3D modeling,
offering a basic framework for future advancements in scene generation and
animation.
65320d35ef70d3555f46f4b6
Chunyi Sun
admin_assigned
2023-10-20T08:53:05.682Z
646dbb124ad7f907279dd18b
/avatars/b92e76ad7e1e9afdc6fe352584633cea.svg
sun
user
chunyin
65320d35ef70d3555f46f4b7
Junlin Han
admin_assigned
2023-10-20T08:53:12.541Z
636e6ee287545ca5a136b4c3
/avatars/208d32b1202e2da210146027212dbdd3.svg
Junlin Han
user
Junlinh
65320d35ef70d3555f46f4b8
Weijian Deng
admin_assigned
2023-10-20T08:53:19.882Z
6431852ddd466752c73d7639
/avatars/86d897afdba81bc45ec3946f622e9dcb.svg
weijian Deng
user
laonafahaodange
65320d35ef70d3555f46f4b9
Xinlong Wang
admin_assigned
2023-10-20T08:53:26.662Z
63ca558304c979828311c5a5
/avatars/2a439d79fba2f987cabe780d10c94d25.svg
Xinlong Wang
user
xinlongwang
65320d35ef70d3555f46f4ba
Zishan Qin
claimed_verified
2023-10-24T07:30:00.761Z
6537435553b7eb25cef80f82
/avatars/b5e395e598e0b34f03a65d42473d69d1.svg
Zishan Qin
user
taylorqin
65320d35ef70d3555f46f4bb
Stephen Gould
admin_assigned
2023-10-20T08:53:42.305Z
63114d9c1f7088709e49b44c
/avatars/8515edaef9c3f46ccd4c1e2efa763c98.svg
Stephen Gould
user
sgould
3D-GPT: Procedural 3D Modeling with Large Language Models
In the pursuit of efficient automated content creation, procedural
generation, leveraging modifiable parameters and rule-based systems, emerges as
a promising approach. Nonetheless, it could be a demanding endeavor, given its
intricate nature necessitating a deep understanding of rules, algorithms, and
parameters. To reduce workload, we introduce 3D-GPT, a framework utilizing
large language models~(LLMs) for instruction-driven 3D modeling. 3D-GPT
positions LLMs as proficient problem solvers, dissecting the procedural 3D
modeling tasks into accessible segments and appointing the apt agent for each
task. 3D-GPT integrates three core agents: the task dispatch agent, the
conceptualization agent, and the modeling agent. They collaboratively achieve
two objectives. First, it enhances concise initial scene descriptions, evolving
them into detailed forms while dynamically adapting the text based on
subsequent instructions. Second, it integrates procedural generation,
extracting parameter values from enriched text to effortlessly interface with
3D software for asset creation. Our empirical investigations confirm that
3D-GPT not only interprets and executes instructions, delivering reliable
results but also collaborates effectively with human designers. Furthermore, it
seamlessly integrates with Blender, unlocking expanded manipulation
possibilities. Our work highlights the potential of LLMs in 3D modeling,
offering a basic framework for future advancements in scene generation and
animation.
3D-GPT leverages large language models to drive 3D modeling tasks, enhancing scene descriptions and interfacing with 3D software through integrated agents.
large language models (LLMs) instruction-driven 3D modeling task dispatch agent conceptualization agent modeling agent procedural generation Blender