Improving Context Fidelity via Native Retrieval-Augmented Reasoning
Large language models (LLMs) often struggle with context fidelity, producing
inconsistent answers when responding to questions based on provided
information. Existing approaches either rely on expensive supervised
fine-tuning to generate evidence post-answer or train models to perform web
searches without necessarily improving utilization of the given context. We
propose CARE, a novel native retrieval-augmented reasoning framework that
teaches LLMs to explicitly integrate in-context evidence within their reasoning
process with the model's own retrieval capabilities. Our method requires
limited labeled evidence data while significantly enhancing both retrieval
accuracy and answer generation performance through strategically retrieved
in-context tokens in the reasoning chain. Extensive experiments on multiple
real-world and counterfactual QA benchmarks demonstrate that our approach
substantially outperforms supervised fine-tuning, traditional
retrieval-augmented generation methods, and external retrieval solutions. This
work represents a fundamental advancement in making LLMs more accurate,
reliable, and efficient for knowledge-intensive tasks.
68cb78015a7803ff3be42de0
Suyuchen Wang
claimed_verified
2025-09-18T13:26:42.099Z
62bb1e0f3ff437e49a3088e5
https://cdn-avatars.huggingface.co/v1/production/uploads/62bb1e0f3ff437e49a3088e5/MWNanci3x5g780xh-704U.png
Suyuchen Wang
user
sheryc
68cb78015a7803ff3be42de1
Jinlin Wang
68cb78015a7803ff3be42de2
Xinyu Wang
claimed_verified
2025-09-19T06:50:21.378Z
65e5fa6a55ed97ae47253e67
/avatars/363ea68811f5aab72c47637385128346.svg
Xinyu Wang
user
W4ng1204
68cb78015a7803ff3be42de3
Shiqi Li
claimed_verified
2025-09-18T13:26:39.471Z
641b26e61911d3be6743e8a4
/avatars/b56668801ebf3e02274ac61bdb83c78a.svg
siky L
user
siky
68cb78015a7803ff3be42de4
Xiangru Tang
68cb78015a7803ff3be42de5
Sirui Hong
68cb78015a7803ff3be42de6
Xiao-Wen Chang
68cb78015a7803ff3be42de7
Chenglin Wu
68cb78015a7803ff3be42de8
Bang Liu
Improving Context Fidelity via Native Retrieval-Augmented Reasoning
Large language models (LLMs) often struggle with context fidelity, producing
inconsistent answers when responding to questions based on provided
information. Existing approaches either rely on expensive supervised
fine-tuning to generate evidence post-answer or train models to perform web
searches without necessarily improving utilization of the given context. We
propose CARE, a novel native retrieval-augmented reasoning framework that
teaches LLMs to explicitly integrate in-context evidence within their reasoning
process with the model's own retrieval capabilities. Our method requires
limited labeled evidence data while significantly enhancing both retrieval
accuracy and answer generation performance through strategically retrieved
in-context tokens in the reasoning chain. Extensive experiments on multiple
real-world and counterfactual QA benchmarks demonstrate that our approach
substantially outperforms supervised fine-tuning, traditional
retrieval-augmented generation methods, and external retrieval solutions. This
work represents a fundamental advancement in making LLMs more accurate,
reliable, and efficient for knowledge-intensive tasks.
CARE, a retrieval-augmented reasoning framework, enhances LLMs by integrating in-context evidence, improving retrieval accuracy and answer generation performance.
LLMs context fidelity supervised fine-tuning evidence post-answer web searches native retrieval-augmented reasoning framework in-context evidence retrieval capabilities labeled evidence data retrieval accuracy answer generation performance reasoning chain QA benchmarks retrieval-augmented generation methods external retrieval solutions