LongCite: Enabling LLMs to Generate Fine-grained Citations in
  Long-context QA
Though current long-context large language models (LLMs) have demonstrated
impressive capacities in answering user questions based on extensive text, the
lack of citations in their responses makes user verification difficult, leading
to concerns about their trustworthiness due to their potential hallucinations.
In this work, we aim to enable long-context LLMs to generate responses with
fine-grained sentence-level citations, improving their faithfulness and
verifiability. We first introduce LongBench-Cite, an automated benchmark for
assessing current LLMs' performance in Long-Context Question Answering with
Citations (LQAC), revealing considerable room for improvement. To this end, we
propose CoF (Coarse to Fine), a novel pipeline that utilizes off-the-shelf LLMs
to automatically generate long-context QA instances with precise sentence-level
citations, and leverage this pipeline to construct LongCite-45k, a large-scale
SFT dataset for LQAC. Finally, we train LongCite-8B and LongCite-9B using the
LongCite-45k dataset, successfully enabling their generation of accurate
responses and fine-grained sentence-level citations in a single output. The
evaluation results on LongBench-Cite show that our trained models achieve
state-of-the-art citation quality, surpassing advanced proprietary models
including GPT-4o.
66d90af260730595a869726c
jiajie Zhang
claimed_verified
2024-09-05T07:24:01.604Z
66cdd285c51a915bd5f2d017
/avatars/14e5794307e4672b1b51d26b31227e0f.svg
Jiajie Zhang
user
NeoZ123
66d90af260730595a869726d
Yushi Bai
admin_assigned
2024-09-05T07:35:53.785Z
64ed568ccf6118a9379a61b8
/avatars/6d040cbcb4a9b624cbe64c9d01cd5c88.svg
Yushi Bai
user
bys0318
66d90af260730595a869726e
Xin Lv
claimed_verified
2024-09-05T07:23:59.827Z
648c64829c935db2b527a764
https://cdn-avatars.huggingface.co/v1/production/uploads/648c64829c935db2b527a764/AnTu1dvpac3hUxLQo8A8K.jpeg
Xin Lv
user
davidlvxin
66d90af260730595a869726f
Wanjun Gu
admin_assigned
2024-09-05T07:35:59.620Z
632d2e46fadf3a596468a74c
https://cdn-avatars.huggingface.co/v1/production/uploads/632d2e46fadf3a596468a74c/NC_3VOrIxP0oKm_vszP1y.jpeg
Wanjun Gu
user
Broccolito
66d90af260730595a8697270
Danqing Liu
admin_assigned
2024-09-05T07:36:05.613Z
64ad519f714bad736d49703c
/avatars/bca409d77ec8ff39abb1f6baaa1d0281.svg
Danqing Liu
user
LZDQ
66d90af260730595a8697271
Minhao Zou
admin_assigned
2024-09-05T07:36:11.662Z
65e57d114230174d546cd5ee
/avatars/42fb7c3bcb5bfda24d8f392fb36ede34.svg
minhao zou
user
zoumh21
66d90af260730595a8697272
Shulin Cao
admin_assigned
2024-09-05T07:36:22.153Z
62c924a6334a6ee11c2e8dfa
/avatars/3dbc37af162b94d68cb83665ac4528c3.svg
ShulinCao
user
caoshulin
66d90af260730595a8697273
Lei Hou
66d90af260730595a8697274
Yuxiao Dong
admin_assigned
2024-09-05T07:36:33.097Z
640e73bdfdeaae1390857b62
/avatars/cd6779e30f716002a7838ed93d5c0754.svg
Yuxiao Dong
user
yuxiaod
66d90af260730595a8697275
Ling Feng
66d90af260730595a8697276
Juanzi Li
admin_assigned
2024-09-05T07:36:40.318Z
65df8cbc2705d9672f55d1aa
/avatars/63e46f15bb76bd9d4508fd0f54f39829.svg
Juanzi Li
user
juanli
LongCite: Enabling LLMs to Generate Fine-grained Citations in
  Long-context QA
Though current long-context large language models (LLMs) have demonstrated
impressive capacities in answering user questions based on extensive text, the
lack of citations in their responses makes user verification difficult, leading
to concerns about their trustworthiness due to their potential hallucinations.
In this work, we aim to enable long-context LLMs to generate responses with
fine-grained sentence-level citations, improving their faithfulness and
verifiability. We first introduce LongBench-Cite, an automated benchmark for
assessing current LLMs' performance in Long-Context Question Answering with
Citations (LQAC), revealing considerable room for improvement. To this end, we
propose CoF (Coarse to Fine), a novel pipeline that utilizes off-the-shelf LLMs
to automatically generate long-context QA instances with precise sentence-level
citations, and leverage this pipeline to construct LongCite-45k, a large-scale
SFT dataset for LQAC. Finally, we train LongCite-8B and LongCite-9B using the
LongCite-45k dataset, successfully enabling their generation of accurate
responses and fine-grained sentence-level citations in a single output. The
evaluation results on LongBench-Cite show that our trained models achieve
state-of-the-art citation quality, surpassing advanced proprietary models
including GPT-4o.
A new pipeline and dataset enable long-context LLMs to generate responses with precise sentence-level citations, improving their trustworthiness and outperforming existing models.
long-context LLMs sentence-level citations LongBench-Cite CoF LongCite-45k LongCite-8B LongCite-9B Long-Context Question Answering with Citations