RetroLLM: Empowering Large Language Models to Retrieve Fine-grained
  Evidence within Generation
Large language models (LLMs) exhibit remarkable generative capabilities but
often suffer from hallucinations. Retrieval-augmented generation (RAG) offers
an effective solution by incorporating external knowledge, but existing methods
still face several limitations: additional deployment costs of separate
retrievers, redundant input tokens from retrieved text chunks, and the lack of
joint optimization of retrieval and generation. To address these issues, we
propose RetroLLM, a unified framework that integrates retrieval and
generation into a single, cohesive process, enabling LLMs to directly generate
fine-grained evidence from the corpus with constrained decoding. Moreover, to
mitigate false pruning in the process of constrained evidence generation, we
introduce (1) hierarchical FM-Index constraints, which generate
corpus-constrained clues to identify a subset of relevant documents before
evidence generation, reducing irrelevant decoding space; and (2) a
forward-looking constrained decoding strategy, which considers the relevance of
future sequences to improve evidence accuracy. Extensive experiments on five
open-domain QA datasets demonstrate RetroLLM's superior performance across both
in-domain and out-of-domain tasks. The code is available at
https://github.com/sunnynexus/RetroLLM.
6760ec669e797e610c5979e0
Xiaoxi Li
admin_assigned
2024-12-17T09:22:41.868Z
66e03eace17fb5ff054b7686
/avatars/2b739ff11e43dd9e701c647a92617f20.svg
Xiaoxi Li
user
lixiaoxi45
6760ec669e797e610c5979e1
Jiajie Jin
admin_assigned
2024-12-17T09:21:52.824Z
6695f14df0ffd8e3a379ad61
/avatars/5ebb7e55ee9c2d93850b279f440675b0.svg
Jiajie Jin
user
jinjiajie
6760ec669e797e610c5979e2
Yujia Zhou
6760ec669e797e610c5979e3
Yongkang Wu
admin_assigned
2024-12-17T09:21:03.390Z
62f3a590261bc5fb2e072a5f
/avatars/d65d362ddc32aca3d6c564252d81e109.svg
YongkangWu
user
wuyongkang
6760ec669e797e610c5979e4
Zhonghua Li
admin_assigned
2024-12-17T09:20:44.760Z
67145068485b75297d6a5ec5
/avatars/67cb59ec072783ee5e4d909c4615bd44.svg
LiZhongHua
user
Benen2024
6760ec669e797e610c5979e5
Qi Ye
6760ec669e797e610c5979e6
Zhicheng Dou
admin_assigned
2024-12-17T09:20:11.013Z
66f0bf59e9d50ec57febf751
/avatars/be97941e60064e5dd806c6fe9db3c537.svg
Zhicheng Dou
user
douzc
RetroLLM: Empowering Large Language Models to Retrieve Fine-grained
  Evidence within Generation
Large language models (LLMs) exhibit remarkable generative capabilities but
often suffer from hallucinations. Retrieval-augmented generation (RAG) offers
an effective solution by incorporating external knowledge, but existing methods
still face several limitations: additional deployment costs of separate
retrievers, redundant input tokens from retrieved text chunks, and the lack of
joint optimization of retrieval and generation. To address these issues, we
propose RetroLLM, a unified framework that integrates retrieval and
generation into a single, cohesive process, enabling LLMs to directly generate
fine-grained evidence from the corpus with constrained decoding. Moreover, to
mitigate false pruning in the process of constrained evidence generation, we
introduce (1) hierarchical FM-Index constraints, which generate
corpus-constrained clues to identify a subset of relevant documents before
evidence generation, reducing irrelevant decoding space; and (2) a
forward-looking constrained decoding strategy, which considers the relevance of
future sequences to improve evidence accuracy. Extensive experiments on five
open-domain QA datasets demonstrate RetroLLM's superior performance across both
in-domain and out-of-domain tasks. The code is available at
https://github.com/sunnynexus/RetroLLM.
RetroLLM is a unified framework that integrates retrieval and generation for LLMs, using constraints and decoding strategies to enhance performance and accuracy in evidence generation.
LLMs hallucinations retrieval-augmented generation (RAG) unified framework constrained decoding hierarchical FM-Index constraints forward-looking constrained decoding strategy extensive experiments open-domain QA datasets