Deliberation on Priors: Trustworthy Reasoning of Large Language Models
  on Knowledge Graphs
Knowledge graph-based retrieval-augmented generation seeks to mitigate
hallucinations in Large Language Models (LLMs) caused by insufficient or
outdated knowledge. However, existing methods often fail to fully exploit the
prior knowledge embedded in knowledge graphs (KGs), particularly their
structural information and explicit or implicit constraints. The former can
enhance the faithfulness of LLMs' reasoning, while the latter can improve the
reliability of response generation. Motivated by these, we propose a
trustworthy reasoning framework, termed Deliberation over Priors (DP), which
sufficiently utilizes the priors contained in KGs. Specifically, DP adopts a
progressive knowledge distillation strategy that integrates structural priors
into LLMs through a combination of supervised fine-tuning and Kahneman-Tversky
optimization, thereby improving the faithfulness of relation path generation.
Furthermore, our framework employs a reasoning-introspection strategy, which
guides LLMs to perform refined reasoning verification based on extracted
constraint priors, ensuring the reliability of response generation. Extensive
experiments on three benchmark datasets demonstrate that DP achieves new
state-of-the-art performance, especially a Hit@1 improvement of 13% on the
ComplexWebQuestions dataset, and generates highly trustworthy responses. We
also conduct various analyses to verify its flexibility and practicality. The
code is available at https://github.com/reml-group/Deliberation-on-Priors.
682e84d83291b134b3370184
Jie Ma
682e84d83291b134b3370185
Ning Qu
claimed_verified
2025-05-22T07:16:55.205Z
653d0136d6f7982a7a52054d
/avatars/5d47d022dd5954a4c3d243a7f0292d32.svg
QUNING
user
stillqu
682e84d83291b134b3370186
Zhitao Gao
claimed_verified
2025-05-26T08:22:25.806Z
673c80505398ab701feb8c50
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/OQ-4gDCZ61eJ8ERX72tnH.png
ZhitaoGao
user
ZhitaoGao
682e84d83291b134b3370187
Rui Xing
682e84d83291b134b3370188
Jun Liu
682e84d83291b134b3370189
Hongbin Pei
682e84d83291b134b337018a
Jiang Xie
682e84d83291b134b337018b
Linyun Song
682e84d83291b134b337018c
Pinghui Wang
682e84d83291b134b337018d
Jing Tao
682e84d83291b134b337018e
Zhou Su
Deliberation on Priors: Trustworthy Reasoning of Large Language Models
  on Knowledge Graphs
Knowledge graph-based retrieval-augmented generation seeks to mitigate
hallucinations in Large Language Models (LLMs) caused by insufficient or
outdated knowledge. However, existing methods often fail to fully exploit the
prior knowledge embedded in knowledge graphs (KGs), particularly their
structural information and explicit or implicit constraints. The former can
enhance the faithfulness of LLMs' reasoning, while the latter can improve the
reliability of response generation. Motivated by these, we propose a
trustworthy reasoning framework, termed Deliberation over Priors (DP), which
sufficiently utilizes the priors contained in KGs. Specifically, DP adopts a
progressive knowledge distillation strategy that integrates structural priors
into LLMs through a combination of supervised fine-tuning and Kahneman-Tversky
optimization, thereby improving the faithfulness of relation path generation.
Furthermore, our framework employs a reasoning-introspection strategy, which
guides LLMs to perform refined reasoning verification based on extracted
constraint priors, ensuring the reliability of response generation. Extensive
experiments on three benchmark datasets demonstrate that DP achieves new
state-of-the-art performance, especially a Hit@1 improvement of 13% on the
ComplexWebQuestions dataset, and generates highly trustworthy responses. We
also conduct various analyses to verify its flexibility and practicality. The
code is available at https://github.com/reml-group/Deliberation-on-Priors.
The Deliberation over Priors framework enhances the trustworthiness of LLMs by integrating structural and constraint priors from knowledge graphs through knowledge distillation and reasoning introspection.
knowledge graph-based retrieval-augmented generation Large Language Models (LLMs) knowledge graphs (KGs) structural information explicit constraints implicit constraints trustworthworthy reasoning framework Deliberation over Priors (DP) progressive knowledge distillation supervised fine-tuning Kahneman-Tversky optimization relation path generation reasoning-introspection strategy ComplexWebQuestions dataset Hit@1 improvement