Knowledge Homophily in Large Language Models
Large Language Models (LLMs) have been increasingly studied as neural
knowledge bases for supporting knowledge-intensive applications such as
question answering and fact checking. However, the structural organization of
their knowledge remains unexplored. Inspired by cognitive neuroscience
findings, such as semantic clustering and priming, where knowing one fact
increases the likelihood of recalling related facts, we investigate an
analogous knowledge homophily pattern in LLMs. To this end, we map LLM
knowledge into a graph representation through knowledge checking at both the
triplet and entity levels. After that, we analyze the knowledgeability
relationship between an entity and its neighbors, discovering that LLMs tend to
possess a similar level of knowledge about entities positioned closer in the
graph. Motivated by this homophily principle, we propose a Graph Neural Network
(GNN) regression model to estimate entity-level knowledgeability scores for
triplets by leveraging their neighborhood scores. The predicted
knowledgeability enables us to prioritize checking less well-known triplets,
thereby maximizing knowledge coverage under the same labeling budget. This not
only improves the efficiency of active labeling for fine-tuning to inject
knowledge into LLMs but also enhances multi-hop path retrieval in
reasoning-intensive question answering.
68dc9a964159d1f2418f9b83
Utkarsh Sahu
68dc9a964159d1f2418f9b84
Zhisheng Qi
68dc9a964159d1f2418f9b85
Mahantesh Halappanavar
68dc9a964159d1f2418f9b86
Nedim Lipka
68dc9a964159d1f2418f9b87
Ryan A. Rossi
68dc9a964159d1f2418f9b88
Franck Dernoncourt
claimed_verified
2025-10-01T10:21:35.032Z
62c5947524171688a9feb992
/avatars/5a151713b9eae8dc566f5957acee3475.svg
Franck Dernoncourt
user
Franck-Dernoncourt
68dc9a964159d1f2418f9b89
Yu Zhang
68dc9a964159d1f2418f9b8a
Yao Ma
68dc9a964159d1f2418f9b8b
Yu Wang
Knowledge Homophily in Large Language Models
Large Language Models (LLMs) have been increasingly studied as neural
knowledge bases for supporting knowledge-intensive applications such as
question answering and fact checking. However, the structural organization of
their knowledge remains unexplored. Inspired by cognitive neuroscience
findings, such as semantic clustering and priming, where knowing one fact
increases the likelihood of recalling related facts, we investigate an
analogous knowledge homophily pattern in LLMs. To this end, we map LLM
knowledge into a graph representation through knowledge checking at both the
triplet and entity levels. After that, we analyze the knowledgeability
relationship between an entity and its neighbors, discovering that LLMs tend to
possess a similar level of knowledge about entities positioned closer in the
graph. Motivated by this homophily principle, we propose a Graph Neural Network
(GNN) regression model to estimate entity-level knowledgeability scores for
triplets by leveraging their neighborhood scores. The predicted
knowledgeability enables us to prioritize checking less well-known triplets,
thereby maximizing knowledge coverage under the same labeling budget. This not
only improves the efficiency of active labeling for fine-tuning to inject
knowledge into LLMs but also enhances multi-hop path retrieval in
reasoning-intensive question answering.
Graph Neural Network regression models estimate entity-level knowledgeability in Large Language Models to improve active labeling and multi-hop reasoning.
Large Language Models knowledge homophily graph representation knowledge checking triplet entity Graph Neural Network knowledgeability scores active labeling multi-hop path retrieval