ReSearch: Learning to Reason with Search for LLMs via Reinforcement
  Learning
Large Language Models (LLMs) have shown remarkable capabilities in reasoning,
exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating
reasoning with external search processes remains challenging, especially for
complex multi-hop questions requiring multiple retrieval steps. We propose
ReSearch, a novel framework that trains LLMs to Reason with Search via
reinforcement learning without using any supervised data on reasoning steps.
Our approach treats search operations as integral components of the reasoning
chain, where when and how to perform searches is guided by text-based thinking,
and search results subsequently influence further reasoning. We train ReSearch
on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct
extensive experiments. Despite being trained on only one dataset, our models
demonstrate strong generalizability across various benchmarks. Analysis reveals
that ReSearch naturally elicits advanced reasoning capabilities such as
reflection and self-correction during the reinforcement learning process.
67e365b0dcfc2aeae1bf3da2
Mingyang Chen
claimed_verified
2025-09-03T08:31:47.108Z
66254042ea4f4ed066a77a1e
/avatars/c749b77417431bb364f6fa2189eabaa2.svg
Mingyang Chen
user
anselcmy
67e365b0dcfc2aeae1bf3da3
Tianpeng Li
claimed_verified
2025-09-03T08:31:45.301Z
646c3ced3e2a7b06594bbaa4
/avatars/6e2d0e2f35e159a7832919a454583ab1.svg
李天鹏
user
yuanshuai
67e365b0dcfc2aeae1bf3da4
Haoze Sun
claimed_verified
2025-05-15T10:41:04.653Z
6436bb0dd58a5ea528c55acb
/avatars/df17b66780e14e07bbe4625f068a94ad.svg
Alvin Sun
user
AlvinSunYooo
67e365b0dcfc2aeae1bf3da5
Yijie Zhou
claimed_verified
2025-09-03T08:31:40.752Z
658670184f349f95cf7d2252
https://cdn-avatars.huggingface.co/v1/production/uploads/658670184f349f95cf7d2252/MfYwxDS1w2kIvav2GvE_U.jpeg
Jie
user
Jayok6
67e365b0dcfc2aeae1bf3da6
Chenzheng Zhu
67e365b0dcfc2aeae1bf3da7
Fan Yang
claimed_verified
2025-09-03T08:31:43.334Z
641c45c921964f8f6d451d16
/avatars/da06cc603f8f9ee46ddb7dc72aae5bec.svg
FanYang
user
fairyang
67e365b0dcfc2aeae1bf3da8
Zenan Zhou
67e365b0dcfc2aeae1bf3da9
Weipeng Chen
67e365b0dcfc2aeae1bf3daa
Haofen Wang
67e365b0dcfc2aeae1bf3dab
Jeff Z. Pan
67e365b0dcfc2aeae1bf3dac
Wen Zhang
67e365b0dcfc2aeae1bf3dad
Huajun Chen
ReSearch: Learning to Reason with Search for LLMs via Reinforcement
  Learning
Large Language Models (LLMs) have shown remarkable capabilities in reasoning,
exemplified by the success of OpenAI-o1 and DeepSeek-R1. However, integrating
reasoning with external search processes remains challenging, especially for
complex multi-hop questions requiring multiple retrieval steps. We propose
ReSearch, a novel framework that trains LLMs to Reason with Search via
reinforcement learning without using any supervised data on reasoning steps.
Our approach treats search operations as integral components of the reasoning
chain, where when and how to perform searches is guided by text-based thinking,
and search results subsequently influence further reasoning. We train ReSearch
on Qwen2.5-7B(-Instruct) and Qwen2.5-32B(-Instruct) models and conduct
extensive experiments. Despite being trained on only one dataset, our models
demonstrate strong generalizability across various benchmarks. Analysis reveals
that ReSearch naturally elicits advanced reasoning capabilities such as
reflection and self-correction during the reinforcement learning process.
ReSearch is a framework that trains LLMs to integrate reasoning with search using reinforcement learning, enhancing capabilities for complex multi-hop questions.
Large Language Models LLMs OpenAI-o1 DeepSeek-R1 ReSearch reinforcement learning Qwen2.5-7B Qwen2.5-32B