LBMKGC: Large Model-Driven Balanced Multimodal Knowledge Graph Completion
渊 过, Qian Ma, Hui Li, Qiao Ning, Furui Zhan, Yu Gu, Ge Yu, Shikai Guo
Multi-modal Knowledge Graph Completion (MMKGC) aims to predict missing entities, relations, or attributes in knowledge graphs by collaboratively modeling the triple structure and multimodal information (e.g., text, images, videos) associated with entities.This approach facilitates the automatic discovery of previously unobserved factual knowledge.However, existing MMKGC methods encounter several critical challenges: (i) the imbalance of inter-entity information across different modalities; (ii) the heterogeneity of intra-entity multimodal information; and (iii) the inconsistency in the volume of information represented by various modalities for a given entity.In this paper, we propose a novel $\underline{L}$arge model-driven $\underline{B}$alanced $\underline{M}$ultimodal $\underline{K}$nowledge $\underline{G}r$aph $\underline{C}$ompletion framework, termed LBMKGC.Initially, LBMKGC employs the Stable Diffusion XL (a large language model) to augment the imbalanced information across modalities. Subsequently, to bridge the semantic gap between heterogeneous modalities, LBMKGC aligns the multimodal embeddings of entities semantically by using the CLIP (Contrastive Language-Image Pre-Training) model.Furthermore, by distinguishing between the perceptual and conceptual attributes of entities, LBMKGC learns entity representations through the adaptive fusion of embeddings from various modalities, guided by structural information.Finally, extensive experiments conducted against 21 state-of-the-art baselines demonstrate that LBMKGC achieves superior performance across diverse datasets and scenarios while maintaining efficiency and generalizability.Our code and data are publicly available at: https://github.com/guoynow/LBMKGC.