ECoRAG: Evidentiality-guided Compression for Long Context RAG
Large Language Models (LLMs) have shown remarkable performance in Open-Domain
Question Answering (ODQA) by leveraging external documents through
Retrieval-Augmented Generation (RAG). To reduce RAG overhead, from longer
context, context compression is necessary. However, prior compression methods
do not focus on filtering out non-evidential information, which limit the
performance in LLM-based RAG. We thus propose Evidentiality-guided RAG, or
ECoRAG framework. ECoRAG improves LLM performance by compressing retrieved
documents based on evidentiality, ensuring whether answer generation is
supported by the correct evidence. As an additional step, ECoRAG reflects
whether the compressed content provides sufficient evidence, and if not,
retrieves more until sufficient. Experiments show that ECoRAG improves LLM
performance on ODQA tasks, outperforming existing compression methods.
Furthermore, ECoRAG is highly cost-efficient, as it not only reduces latency
but also minimizes token usage by retaining only the necessary information to
generate the correct answer. Code is available at
https://github.com/ldilab/ECoRAG.
68468cb23ec10bdd8ab4db5b
Yeonseok Jeong
claimed_verified
2025-06-09T10:11:11.745Z
645aedd221ab438e732bff43
/avatars/31e14fee670fd5ddd296ea0249dbf710.svg
Yeonseok Jeong
user
yeonseokjeong
68468cb23ec10bdd8ab4db5c
Jinsu Kim
68468cb23ec10bdd8ab4db5d
Dohyeon Lee
68468cb23ec10bdd8ab4db5e
Seung-won Hwang
ECoRAG: Evidentiality-guided Compression for Long Context RAG
Large Language Models (LLMs) have shown remarkable performance in Open-Domain
Question Answering (ODQA) by leveraging external documents through
Retrieval-Augmented Generation (RAG). To reduce RAG overhead, from longer
context, context compression is necessary. However, prior compression methods
do not focus on filtering out non-evidential information, which limit the
performance in LLM-based RAG. We thus propose Evidentiality-guided RAG, or
ECoRAG framework. ECoRAG improves LLM performance by compressing retrieved
documents based on evidentiality, ensuring whether answer generation is
supported by the correct evidence. As an additional step, ECoRAG reflects
whether the compressed content provides sufficient evidence, and if not,
retrieves more until sufficient. Experiments show that ECoRAG improves LLM
performance on ODQA tasks, outperforming existing compression methods.
Furthermore, ECoRAG is highly cost-efficient, as it not only reduces latency
but also minimizes token usage by retaining only the necessary information to
generate the correct answer. Code is available at
https://github.com/ldilab/ECoRAG.
ECoRAG framework enhances LLM performance in ODQA by compressing retrieved documents based on evidentiality, reducing latency and token usage.
Retrieval-Augmented Generation (RAG) context compression evidentiality LLM Open-Domain Question Answering (ODQA)