LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
In traditional RAG framework, the basic retrieval units are normally short.
The common retrievers like DPR normally work with 100-word Wikipedia
paragraphs. Such a design forces the retriever to search over a large corpus to
find the `needle' unit. In contrast, the readers only need to extract answers
from the short retrieved units. Such an imbalanced `heavy' retriever and
`light' reader design can lead to sub-optimal performance. In order to
alleviate the imbalance, we propose a new framework LongRAG, consisting of a
`long retriever' and a `long reader'. LongRAG processes the entire Wikipedia
into 4K-token units, which is 30x longer than before. By increasing the unit
size, we significantly reduce the total units from 22M to 700K. This
significantly lowers the burden of retriever, which leads to a remarkable
retrieval score: answer recall@1=71% on NQ (previously 52%) and answer
recall@2=72% (previously 47%) on HotpotQA (full-wiki). Then we feed the top-k
retrieved units (approx 30K tokens) to an existing long-context LLM to
perform zero-shot answer extraction. Without requiring any training, LongRAG
achieves an EM of 62.7% on NQ, which is the best known result. LongRAG also
achieves 64.3% on HotpotQA (full-wiki), which is on par of the SoTA model. Our
study offers insights into the future roadmap for combining RAG with
long-context LLMs.
6678e77c66539bc0a08db180
Ziyan Jiang
extracted_confirmed
2024-06-24T03:32:43.711Z
64778fb8168cb428e00f69b0
https://cdn-avatars.huggingface.co/v1/production/uploads/64778fb8168cb428e00f69b0/D_XYg74zHG9K3HUJj_gD4.jpeg
Ziyan Jiang
user
ziyjiang
6678e77c66539bc0a08db181
Xueguang Ma
extracted_confirmed
2024-06-26T18:51:44.209Z
5ec82854968f6028e0559f70
/avatars/45b58d912f7d00cb351947cd79d5eeb4.svg
Xueguang Ma
user
MrLight
6678e77c66539bc0a08db182
Wenhu Chen
extracted_pending
2024-06-24T03:26:53.799Z
6313a86154e6e5d9f0f94e04
https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
Wenhu Chen
user
wenhu
LongRAG: Enhancing Retrieval-Augmented Generation with Long-context LLMs
In traditional RAG framework, the basic retrieval units are normally short.
The common retrievers like DPR normally work with 100-word Wikipedia
paragraphs. Such a design forces the retriever to search over a large corpus to
find the `needle' unit. In contrast, the readers only need to extract answers
from the short retrieved units. Such an imbalanced `heavy' retriever and
`light' reader design can lead to sub-optimal performance. In order to
alleviate the imbalance, we propose a new framework LongRAG, consisting of a
`long retriever' and a `long reader'. LongRAG processes the entire Wikipedia
into 4K-token units, which is 30x longer than before. By increasing the unit
size, we significantly reduce the total units from 22M to 700K. This
significantly lowers the burden of retriever, which leads to a remarkable
retrieval score: answer recall@1=71% on NQ (previously 52%) and answer
recall@2=72% (previously 47%) on HotpotQA (full-wiki). Then we feed the top-k
retrieved units (approx 30K tokens) to an existing long-context LLM to
perform zero-shot answer extraction. Without requiring any training, LongRAG
achieves an EM of 62.7% on NQ, which is the best known result. LongRAG also
achieves 64.3% on HotpotQA (full-wiki), which is on par of the SoTA model. Our
study offers insights into the future roadmap for combining RAG with
long-context LLMs.
The LongRAG framework improves information retrieval and answer extraction by using longer retrieval units and leveraging long-context LLMs, leading to better performance on benchmark datasets.
LongRAG retriever reader 4K-token units long-context LLMs answer recall@1 answer recall@2 EM NQ HotpotQA full-wiki