ReasonIR: Training Retrievers for Reasoning Tasks
We present ReasonIR-8B, the first retriever specifically trained for general
reasoning tasks. Existing retrievers have shown limited gains on reasoning
tasks, in part because existing training datasets focus on short factual
queries tied to documents that straightforwardly answer them. We develop a
synthetic data generation pipeline that, for each document, our pipeline
creates a challenging and relevant query, along with a plausibly related but
ultimately unhelpful hard negative. By training on a mixture of our synthetic
data and existing public data, ReasonIR-8B achieves a new state-of-the-art of
29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a
widely-used reasoning-intensive information retrieval (IR) benchmark. When
applied to RAG tasks, ReasonIR-8B improves MMLU and GPQA performance by 6.4%
and 22.6% respectively, relative to the closed-book baseline, outperforming
other retrievers and search engines. In addition, ReasonIR-8B uses test-time
compute more effectively: on BRIGHT, its performance consistently increases
with longer and more information-rich rewritten queries; it continues to
outperform other retrievers when combined with an LLM reranker. Our training
recipe is general and can be easily extended to future LLMs; to this end, we
open-source our code, data, and model.
68118a9f4570c2ba44bf4418
Rulin Shao
admin_assigned
2025-04-30T10:02:20.688Z
6334a0bd31a2be3938c59537
https://cdn-avatars.huggingface.co/v1/production/uploads/6334a0bd31a2be3938c59537/kSetFUWAmJbPQ1KSlNKBr.jpeg
Rulin Shao
user
rulins
68118a9f4570c2ba44bf4419
Rui Qiao
claimed_verified
2025-05-02T06:37:39.498Z
6564aa8b86bb2e3c8c093f8d
https://cdn-avatars.huggingface.co/v1/production/uploads/6564aa8b86bb2e3c8c093f8d/opXj6iCQqdSiE9POu9o4u.jpeg
Rui Qiao
user
qiaoruiyt
68118a9f4570c2ba44bf441a
Varsha Kishore
68118a9f4570c2ba44bf441b
Niklas Muennighoff
admin_assigned
2025-04-30T10:02:27.811Z
5f1eb362eec0ad2a071ad6e2
https://cdn-avatars.huggingface.co/v1/production/uploads/5f1eb362eec0ad2a071ad6e2/IXMYkYKuTwn6kBdWnQeeY.png
Niklas Muennighoff
user
Muennighoff
68118a9f4570c2ba44bf441c
Xi Victoria Lin
68118a9f4570c2ba44bf441d
Daniela Rus
68118a9f4570c2ba44bf441e
Bryan Kian Hsiang Low
68118a9f4570c2ba44bf441f
Sewon Min
admin_assigned
2025-04-30T10:01:46.233Z
63a76d0de27a6dbd485fe863
https://cdn-avatars.huggingface.co/v1/production/uploads/63a76d0de27a6dbd485fe863/qJJwHOuvyQGq1o0KscOF_.jpeg
Sewon Min
user
sewon
68118a9f4570c2ba44bf4420
Wen-tau Yih
68118a9f4570c2ba44bf4421
Pang Wei Koh
admin_assigned
2025-04-30T10:01:57.373Z
641b4263abfce26bcf7b27de
/avatars/e91b4205e4f74b0dd8c333c23203a924.svg
Pang Wei Koh
user
pangwei
68118a9f4570c2ba44bf4422
Luke Zettlemoyer
ReasonIR: Training Retrievers for Reasoning Tasks
We present ReasonIR-8B, the first retriever specifically trained for general
reasoning tasks. Existing retrievers have shown limited gains on reasoning
tasks, in part because existing training datasets focus on short factual
queries tied to documents that straightforwardly answer them. We develop a
synthetic data generation pipeline that, for each document, our pipeline
creates a challenging and relevant query, along with a plausibly related but
ultimately unhelpful hard negative. By training on a mixture of our synthetic
data and existing public data, ReasonIR-8B achieves a new state-of-the-art of
29.9 nDCG@10 without reranker and 36.9 nDCG@10 with reranker on BRIGHT, a
widely-used reasoning-intensive information retrieval (IR) benchmark. When
applied to RAG tasks, ReasonIR-8B improves MMLU and GPQA performance by 6.4%
and 22.6% respectively, relative to the closed-book baseline, outperforming
other retrievers and search engines. In addition, ReasonIR-8B uses test-time
compute more effectively: on BRIGHT, its performance consistently increases
with longer and more information-rich rewritten queries; it continues to
outperform other retrievers when combined with an LLM reranker. Our training
recipe is general and can be easily extended to future LLMs; to this end, we
open-source our code, data, and model.
ReasonIR-8B, a retriever trained with synthetic challenging queries, achieves state-of-the-art results in reasoning-intensive information retrieval and RAG tasks, improving performance and computational efficiency.
retriever synthetic data generation nDCG@10 BRIGHT RAG tasks MMLU GPQA closed-book baseline test-time compute LLM reranker