CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners
Knowledge Editing (KE) enables the modification of outdated or incorrect
information in large language models (LLMs). While existing KE methods can
update isolated facts, they struggle to generalize these updates to multi-hop
reasoning tasks that depend on the modified knowledge. Through an analysis of
reasoning circuits -- the neural pathways LLMs use for knowledge-based
inference, we observe that current layer-localized KE approaches, such as MEMIT
and WISE, which edit only single or a few model layers, struggle to effectively
incorporate updated information into these reasoning pathways. To address this
limitation, we propose CaKE (Circuit-aware Knowledge Editing), a novel method
that enables more effective integration of updated knowledge in LLMs. CaKE
leverages strategically curated data, guided by our circuits-based analysis,
that enforces the model to utilize the modified knowledge, stimulating the
model to develop appropriate reasoning circuits for newly integrated knowledge.
Experimental results show that CaKE enables more accurate and consistent use of
updated knowledge across related reasoning tasks, leading to an average of 20%
improvement in multi-hop reasoning accuracy on MQuAKE dataset compared to
existing KE methods. We release the code and data in
https://github.com/zjunlp/CaKE.
67dcd18ad2550735425351bf
Yunzhi Yao
admin_assigned
2025-03-21T13:06:47.575Z
6122fbe636a4c36a99dbea7b
/avatars/c0cd2c1ef58e315d9adda9d26000f625.svg
Yunzhi Yao
user
cowTodd
67dcd18ad2550735425351c0
Jizhan Fang
admin_assigned
2025-03-21T13:06:55.484Z
669663472d25bd04e9af1d66
/avatars/8b11d5d79d1b8b205baa498a942f573c.svg
Jizhan Fang
user
JizhanFang
67dcd18ad2550735425351c1
Jia-Chen Gu
admin_assigned
2025-03-21T13:07:01.906Z
677711d332399119a31bed14
/avatars/5648e8520c9ede4db42148f7da84464b.svg
Jia-Chen Gu
user
gujc
67dcd18ad2550735425351c2
Ningyu Zhang
claimed_verified
2025-03-21T11:40:59.405Z
620b3bbb0668e435407c8d0a
/avatars/e0fccbb2577d76088e09f054c35cffbc.svg
Ningyu Zhang
user
Ningyu
67dcd18ad2550735425351c3
Shumin Deng
admin_assigned
2025-03-21T13:07:08.198Z
6441f1d2603214724ec0c1c2
/avatars/d3c4b759e6a5635e37ff715fae52e5ba.svg
Shumin Deng
user
231sm
67dcd18ad2550735425351c4
Huajun Chen
admin_assigned
2025-03-21T13:07:15.045Z
67c06592ebc2e682b2628bde
/avatars/1032c0319eded956d2dd461b0e1b4ddb.svg
Huajun Chen
user
HuajunChen
67dcd18ad2550735425351c5
Nanyun Peng
CaKE: Circuit-aware Editing Enables Generalizable Knowledge Learners
Knowledge Editing (KE) enables the modification of outdated or incorrect
information in large language models (LLMs). While existing KE methods can
update isolated facts, they struggle to generalize these updates to multi-hop
reasoning tasks that depend on the modified knowledge. Through an analysis of
reasoning circuits -- the neural pathways LLMs use for knowledge-based
inference, we observe that current layer-localized KE approaches, such as MEMIT
and WISE, which edit only single or a few model layers, struggle to effectively
incorporate updated information into these reasoning pathways. To address this
limitation, we propose CaKE (Circuit-aware Knowledge Editing), a novel method
that enables more effective integration of updated knowledge in LLMs. CaKE
leverages strategically curated data, guided by our circuits-based analysis,
that enforces the model to utilize the modified knowledge, stimulating the
model to develop appropriate reasoning circuits for newly integrated knowledge.
Experimental results show that CaKE enables more accurate and consistent use of
updated knowledge across related reasoning tasks, leading to an average of 20%
improvement in multi-hop reasoning accuracy on MQuAKE dataset compared to
existing KE methods. We release the code and data in
https://github.com/zjunlp/CaKE.
CaKE enhances large language models' multi-hop reasoning by targeting specific neural pathways for knowledge editing, improving accuracy compared to existing methods.
knowledge editing large language models multi-hop reasoning reasoning circuits MEMIT WISE circuit-aware knowledge editing knowledge-based inference