Reinforced Internal-External Knowledge Synergistic Reasoning for
  Efficient Adaptive Search Agent
Retrieval-augmented generation (RAG) is a common strategy to reduce
hallucinations in Large Language Models (LLMs). While reinforcement learning
(RL) can enable LLMs to act as search agents by activating retrieval
capabilities, existing ones often underutilize their internal knowledge. This
can lead to redundant retrievals, potential harmful knowledge conflicts, and
increased inference latency. To address these limitations, an efficient and
adaptive search agent capable of discerning optimal retrieval timing and
synergistically integrating parametric (internal) and retrieved (external)
knowledge is in urgent need. This paper introduces the Reinforced
Internal-External Knowledge Synergistic Reasoning Agent (IKEA), which could
indentify its own knowledge boundary and prioritize the utilization of internal
knowledge, resorting to external search only when internal knowledge is deemed
insufficient. This is achieved using a novel knowledge-boundary aware reward
function and a knowledge-boundary aware training dataset. These are designed
for internal-external knowledge synergy oriented RL, incentivizing the model to
deliver accurate answers, minimize unnecessary retrievals, and encourage
appropriate external searches when its own knowledge is lacking. Evaluations
across multiple knowledge reasoning tasks demonstrate that IKEA significantly
outperforms baseline methods, reduces retrieval frequency significantly, and
exhibits robust generalization capabilities.
6822ba9e4867f1e9dc14c294
Ziyang Huang
claimed_verified
2025-05-13T08:11:42.410Z
616648c84c0937d31946f21b
/avatars/7ca27de5c5116c91ff1db61ba6277ed5.svg
Ziyang
user
hzy
6822ba9e4867f1e9dc14c295
Xiaowei Yuan
6822ba9e4867f1e9dc14c296
Yiming Ju
6822ba9e4867f1e9dc14c297
Jun Zhao
6822ba9e4867f1e9dc14c298
Kang Liu
Reinforced Internal-External Knowledge Synergistic Reasoning for
  Efficient Adaptive Search Agent
Retrieval-augmented generation (RAG) is a common strategy to reduce
hallucinations in Large Language Models (LLMs). While reinforcement learning
(RL) can enable LLMs to act as search agents by activating retrieval
capabilities, existing ones often underutilize their internal knowledge. This
can lead to redundant retrievals, potential harmful knowledge conflicts, and
increased inference latency. To address these limitations, an efficient and
adaptive search agent capable of discerning optimal retrieval timing and
synergistically integrating parametric (internal) and retrieved (external)
knowledge is in urgent need. This paper introduces the Reinforced
Internal-External Knowledge Synergistic Reasoning Agent (IKEA), which could
indentify its own knowledge boundary and prioritize the utilization of internal
knowledge, resorting to external search only when internal knowledge is deemed
insufficient. This is achieved using a novel knowledge-boundary aware reward
function and a knowledge-boundary aware training dataset. These are designed
for internal-external knowledge synergy oriented RL, incentivizing the model to
deliver accurate answers, minimize unnecessary retrievals, and encourage
appropriate external searches when its own knowledge is lacking. Evaluations
across multiple knowledge reasoning tasks demonstrate that IKEA significantly
outperforms baseline methods, reduces retrieval frequency significantly, and
exhibits robust generalization capabilities.
The IKEA agent optimizes reinforcement learning in LLMs by synergistically using internal and external knowledge, reducing hallucinations and unnecessary retrievals.
Retrieval-augmented generation Large Language Models reinforcement learning search agent internal knowledge external knowledge knowledge-boundary aware reward function knowledge-boundary aware training dataset internal-external knowledge synergy