Spatial Knowledge Graph-Guided Multimodal Synthesis
Recent advances in multimodal large language models (MLLMs) have
significantly enhanced their capabilities; however, their spatial perception
abilities remain a notable limitation. To address this challenge, multimodal
data synthesis offers a promising solution. Yet, ensuring that synthesized data
adhere to spatial common sense is a non-trivial task. In this work, we
introduce SKG2Data, a novel multimodal synthesis approach guided by spatial
knowledge graphs, grounded in the concept of knowledge-to-data generation.
SKG2Data automatically constructs a Spatial Knowledge Graph (SKG) to emulate
human-like perception of spatial directions and distances, which is
subsequently utilized to guide multimodal data synthesis. Extensive experiments
demonstrate that data synthesized from diverse types of spatial knowledge,
including direction and distance, not only enhance the spatial perception and
reasoning abilities of MLLMs but also exhibit strong generalization
capabilities. We hope that the idea of knowledge-based data synthesis can
advance the development of spatial intelligence.
6837b67765209bfdd650594e
Yida Xue
6837b67765209bfdd650594f
Zhen Bi
6837b67765209bfdd6505950
Jinnan Yang
6837b67765209bfdd6505951
Jungang Lou
6837b67765209bfdd6505952
Huajun Chen
6837b67765209bfdd6505953
Ningyu Zhang
claimed_verified
2025-06-13T07:42:19.330Z
620b3bbb0668e435407c8d0a
/avatars/e0fccbb2577d76088e09f054c35cffbc.svg
Ningyu Zhang
user
Ningyu
Spatial Knowledge Graph-Guided Multimodal Synthesis
Recent advances in multimodal large language models (MLLMs) have
significantly enhanced their capabilities; however, their spatial perception
abilities remain a notable limitation. To address this challenge, multimodal
data synthesis offers a promising solution. Yet, ensuring that synthesized data
adhere to spatial common sense is a non-trivial task. In this work, we
introduce SKG2Data, a novel multimodal synthesis approach guided by spatial
knowledge graphs, grounded in the concept of knowledge-to-data generation.
SKG2Data automatically constructs a Spatial Knowledge Graph (SKG) to emulate
human-like perception of spatial directions and distances, which is
subsequently utilized to guide multimodal data synthesis. Extensive experiments
demonstrate that data synthesized from diverse types of spatial knowledge,
including direction and distance, not only enhance the spatial perception and
reasoning abilities of MLLMs but also exhibit strong generalization
capabilities. We hope that the idea of knowledge-based data synthesis can
advance the development of spatial intelligence.
SKG2Data enhances spatial perception and reasoning in multimodal large language models by using spatial knowledge graphs to guide data synthesis.
multimodal large language models spatial perception multimodal synthesis spatial knowledge graphs knowledge-to-data generation spatial directions spatial distances spatial intelligence