DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for
  Dynamic Reranking in Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) systems combine large language models
(LLMs) with external knowledge retrieval, making them highly effective for
knowledge-intensive tasks. A crucial but often under-explored component of
these systems is the reranker, which refines retrieved documents to enhance
generation quality and explainability. The challenge of selecting the optimal
number of documents (k) remains unsolved: too few may omit critical
information, while too many introduce noise and inefficiencies. Although recent
studies have explored LLM-based rerankers, they primarily leverage internal
model knowledge and overlook the rich supervisory signals that LLMs can
provide, such as using response quality as feedback for optimizing reranking
decisions. In this paper, we propose DynamicRAG, a novel RAG framework where
the reranker dynamically adjusts both the order and number of retrieved
documents based on the query. We model the reranker as an agent optimized
through reinforcement learning (RL), using rewards derived from LLM output
quality. Across seven knowledge-intensive datasets, DynamicRAG demonstrates
superior performance, achieving state-of-the-art results. The model, data and
code are available at https://github.com/GasolSun36/DynamicRAG
6823496167281062097f3ac6
Jiashuo Sun
claimed_verified
2025-05-14T07:35:31.777Z
64527f701a57e1179c1c3693
/avatars/25b2632d7aa9ce26d5d4924ecb00c4f4.svg
Jiashuo Sun
user
gasolsun
6823496167281062097f3ac7
Xianrui Zhong
6823496167281062097f3ac8
Sizhe Zhou
6823496167281062097f3ac9
Jiawei Han
DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for
  Dynamic Reranking in Retrieval-Augmented Generation
Retrieval-augmented generation (RAG) systems combine large language models
(LLMs) with external knowledge retrieval, making them highly effective for
knowledge-intensive tasks. A crucial but often under-explored component of
these systems is the reranker, which refines retrieved documents to enhance
generation quality and explainability. The challenge of selecting the optimal
number of documents (k) remains unsolved: too few may omit critical
information, while too many introduce noise and inefficiencies. Although recent
studies have explored LLM-based rerankers, they primarily leverage internal
model knowledge and overlook the rich supervisory signals that LLMs can
provide, such as using response quality as feedback for optimizing reranking
decisions. In this paper, we propose DynamicRAG, a novel RAG framework where
the reranker dynamically adjusts both the order and number of retrieved
documents based on the query. We model the reranker as an agent optimized
through reinforcement learning (RL), using rewards derived from LLM output
quality. Across seven knowledge-intensive datasets, DynamicRAG demonstrates
superior performance, achieving state-of-the-art results. The model, data and
code are available at https://github.com/GasolSun36/DynamicRAG
DynamicRAG optimizes the retrieval-augmented generation framework by dynamically adjusting document selection and order based on a reinforcement learning agent trained with LLM output quality.
retrieval-augmented generation LLMs reranker reinforcement learning dynamic retrieval document selection LLM output quality