SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and
  Self-Reflective Agent
Indoor scene synthesis has become increasingly important with the rise of
Embodied AI, which requires 3D environments that are not only visually
realistic but also physically plausible and functionally diverse. While recent
approaches have advanced visual fidelity, they often remain constrained to
fixed scene categories, lack sufficient object-level detail and physical
consistency, and struggle to align with complex user instructions. In this
work, we present SceneWeaver, a reflective agentic framework that unifies
diverse scene synthesis paradigms through tool-based iterative refinement. At
its core, SceneWeaver employs a language model-based planner to select from a
suite of extensible scene generation tools, ranging from data-driven generative
models to visual- and LLM-based methods, guided by self-evaluation of physical
plausibility, visual realism, and semantic alignment with user input. This
closed-loop reason-act-reflect design enables the agent to identify semantic
inconsistencies, invoke targeted tools, and update the environment over
successive iterations. Extensive experiments on both common and open-vocabulary
room types demonstrate that SceneWeaver not only outperforms prior methods on
physical, visual, and semantic metrics, but also generalizes effectively to
complex scenes with diverse instructions, marking a step toward general-purpose
3D environment generation. Project website: https://scene-weaver.github.io/.
68d5f3868ccd91bdd39ffd88
Yandan Yang
68d5f3868ccd91bdd39ffd89
Baoxiong Jia
admin_assigned
2025-09-26T16:18:40.402Z
6304b389bad6ce7fc02691d5
/avatars/a762ca59624ce409650165f36b973488.svg
Baoxiong Jia
user
BuzzBeater
68d5f3868ccd91bdd39ffd8a
Shujie Zhang
admin_assigned
2025-09-26T16:18:34.328Z
68b1223b95e5722566d8da61
/avatars/10008327fdbd36f9190ce8a7ddc1efa6.svg
zhang
user
shujiezhang
68d5f3868ccd91bdd39ffd8b
Siyuan Huang
SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and
  Self-Reflective Agent
Indoor scene synthesis has become increasingly important with the rise of
Embodied AI, which requires 3D environments that are not only visually
realistic but also physically plausible and functionally diverse. While recent
approaches have advanced visual fidelity, they often remain constrained to
fixed scene categories, lack sufficient object-level detail and physical
consistency, and struggle to align with complex user instructions. In this
work, we present SceneWeaver, a reflective agentic framework that unifies
diverse scene synthesis paradigms through tool-based iterative refinement. At
its core, SceneWeaver employs a language model-based planner to select from a
suite of extensible scene generation tools, ranging from data-driven generative
models to visual- and LLM-based methods, guided by self-evaluation of physical
plausibility, visual realism, and semantic alignment with user input. This
closed-loop reason-act-reflect design enables the agent to identify semantic
inconsistencies, invoke targeted tools, and update the environment over
successive iterations. Extensive experiments on both common and open-vocabulary
room types demonstrate that SceneWeaver not only outperforms prior methods on
physical, visual, and semantic metrics, but also generalizes effectively to
complex scenes with diverse instructions, marking a step toward general-purpose
3D environment generation. Project website: https://scene-weaver.github.io/.
SceneWeaver, a reflective agentic framework, uses a language model-based planner to iteratively refine 3D scene synthesis, achieving high physical, visual, and semantic quality across diverse instructions.
language model-based planner scene generation tools data-driven generative models visual- and LLM-based methods self-evaluation physical plausibility visual realism semantic alignment closed-loop reason-act-reflect design semantic inconsistencies targeted tools general-purpose 3D environment generation