CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation
  Generation
Retrieval-Augmented Generation (RAG) has become a powerful paradigm for
enhancing large language models (LLMs) through external knowledge retrieval.
Despite its widespread attention, existing academic research predominantly
focuses on single-turn RAG, leaving a significant gap in addressing the
complexities of multi-turn conversations found in real-world applications. To
bridge this gap, we introduce CORAL, a large-scale benchmark designed to assess
RAG systems in realistic multi-turn conversational settings. CORAL includes
diverse information-seeking conversations automatically derived from Wikipedia
and tackles key challenges such as open-domain coverage, knowledge intensity,
free-form responses, and topic shifts. It supports three core tasks of
conversational RAG: passage retrieval, response generation, and citation
labeling. We propose a unified framework to standardize various conversational
RAG methods and conduct a comprehensive evaluation of these methods on CORAL,
demonstrating substantial opportunities for improving existing approaches.
6722ddfb2b7a133fcb7a3978
Yiruo Cheng
claimed_verified
2024-10-31T09:54:45.558Z
6530eba73572cd4fb78daf36
/avatars/4e6f0f07bb4e0ed9e3c58d49eaf78fe5.svg
Yiruo Cheng
user
ariya2357
6722ddfb2b7a133fcb7a3979
Kelong Mao
claimed_verified
2024-12-30T19:37:57.951Z
649a65605c74a2125c22bbc1
/avatars/e7435d3aeeb59acc6f6f43b48d6982a0.svg
Mao
user
kyriemao
6722ddfb2b7a133fcb7a397a
Ziliang Zhao
6722ddfb2b7a133fcb7a397b
Guanting Dong
claimed_verified
2024-10-31T09:54:42.575Z
61cd4b833dd34ba1985e0753
https://cdn-avatars.huggingface.co/v1/production/uploads/61cd4b833dd34ba1985e0753/BfHfrwotoMESpXZOHiIe4.png
KABI
user
dongguanting
6722ddfb2b7a133fcb7a397c
Hongjin Qian
6722ddfb2b7a133fcb7a397d
Yongkang Wu
6722ddfb2b7a133fcb7a397e
Tetsuya Sakai
6722ddfb2b7a133fcb7a397f
Ji-Rong Wen
6722ddfb2b7a133fcb7a3980
Zhicheng Dou
CORAL: Benchmarking Multi-turn Conversational Retrieval-Augmentation
  Generation
Retrieval-Augmented Generation (RAG) has become a powerful paradigm for
enhancing large language models (LLMs) through external knowledge retrieval.
Despite its widespread attention, existing academic research predominantly
focuses on single-turn RAG, leaving a significant gap in addressing the
complexities of multi-turn conversations found in real-world applications. To
bridge this gap, we introduce CORAL, a large-scale benchmark designed to assess
RAG systems in realistic multi-turn conversational settings. CORAL includes
diverse information-seeking conversations automatically derived from Wikipedia
and tackles key challenges such as open-domain coverage, knowledge intensity,
free-form responses, and topic shifts. It supports three core tasks of
conversational RAG: passage retrieval, response generation, and citation
labeling. We propose a unified framework to standardize various conversational
RAG methods and conduct a comprehensive evaluation of these methods on CORAL,
demonstrating substantial opportunities for improving existing approaches.
CORAL is a large-scale benchmark for evaluating Retrieval-Augmented Generation (RAG) systems in multi-turn conversational settings, addressing challenges like open-domain coverage, knowledge intensity, free-form responses, and topic shifts.
Retrieval-Augmented Generation (RAG) large language models (LLMs) multi-turn conversations CORAL passage retrieval response generation citation labeling