ZeroSearch: Incentivize the Search Capability of LLMs without Searching
Effective information searching is essential for enhancing the reasoning and
generation capabilities of large language models (LLMs). Recent research has
explored using reinforcement learning (RL) to improve LLMs' search capabilities
by interacting with live search engines in real-world environments. While these
approaches show promising results, they face two major challenges: (1)
Uncontrolled Document Quality: The quality of documents returned by search
engines is often unpredictable, introducing noise and instability into the
training process. (2) Prohibitively High API Costs: RL training requires
frequent rollouts, potentially involving hundreds of thousands of search
requests, which incur substantial API expenses and severely constrain
scalability. To address these challenges, we introduce ZeroSearch, a
reinforcement learning framework that incentivizes the search capabilities of
LLMs without interacting with real search engines. Our approach begins with
lightweight supervised fine-tuning to transform the LLM into a retrieval module
capable of generating both relevant and noisy documents in response to a query.
During RL training, we employ a curriculum-based rollout strategy that
incrementally degrades the quality of generated documents, progressively
eliciting the model's reasoning ability by exposing it to increasingly
challenging retrieval scenarios. Extensive experiments demonstrate that
ZeroSearch effectively incentivizes the search capabilities of LLMs using a 3B
LLM as the retrieval module. Remarkably, a 7B retrieval module achieves
comparable performance to the real search engine, while a 14B retrieval module
even surpasses it. Furthermore, it generalizes well across both base and
instruction-tuned models of various parameter sizes and is compatible with a
wide range of RL algorithms.
681c15ab84d0a008fcdb1ee8
Hao Sun
claimed_verified
2025-05-09T07:22:41.387Z
662cef87bf294785d1f45c75
/avatars/15c253ae955c86ad2b459b93edbe694b.svg
hao sun
user
sunhaonlp
681c15ab84d0a008fcdb1ee9
Zile Qiao
681c15ab84d0a008fcdb1eea
Jiayan Guo
claimed_verified
2025-05-08T08:59:01.834Z
66224557c61c7fbd98099079
/avatars/a4f2144585c808865c73b5b7f0087c1f.svg
Jiayan Guo
user
SpaceProduct
681c15ab84d0a008fcdb1eeb
Xuanbo Fan
681c15ab84d0a008fcdb1eec
Yingyan Hou
681c15ab84d0a008fcdb1eed
Yong Jiang
681c15ab84d0a008fcdb1eee
Pengjun Xie
681c15ab84d0a008fcdb1eef
Fei Huang
681c15ab84d0a008fcdb1ef0
Yan Zhang
ZeroSearch: Incentivize the Search Capability of LLMs without Searching
Effective information searching is essential for enhancing the reasoning and
generation capabilities of large language models (LLMs). Recent research has
explored using reinforcement learning (RL) to improve LLMs' search capabilities
by interacting with live search engines in real-world environments. While these
approaches show promising results, they face two major challenges: (1)
Uncontrolled Document Quality: The quality of documents returned by search
engines is often unpredictable, introducing noise and instability into the
training process. (2) Prohibitively High API Costs: RL training requires
frequent rollouts, potentially involving hundreds of thousands of search
requests, which incur substantial API expenses and severely constrain
scalability. To address these challenges, we introduce ZeroSearch, a
reinforcement learning framework that incentivizes the search capabilities of
LLMs without interacting with real search engines. Our approach begins with
lightweight supervised fine-tuning to transform the LLM into a retrieval module
capable of generating both relevant and noisy documents in response to a query.
During RL training, we employ a curriculum-based rollout strategy that
incrementally degrades the quality of generated documents, progressively
eliciting the model's reasoning ability by exposing it to increasingly
challenging retrieval scenarios. Extensive experiments demonstrate that
ZeroSearch effectively incentivizes the search capabilities of LLMs using a 3B
LLM as the retrieval module. Remarkably, a 7B retrieval module achieves
comparable performance to the real search engine, while a 14B retrieval module
even surpasses it. Furthermore, it generalizes well across both base and
instruction-tuned models of various parameter sizes and is compatible with a
wide range of RL algorithms.
ZeroSearch is a reinforcement learning framework that improves LLM search capabilities without using real search engines, using supervised fine-tuning and curriculum-based rollouts to handle document quality and API cost challenges.
reinforcement learning LLMs search capabilities document quality API costs lightweight supervised fine-tuning retrieval module curriculum-based rollout strategy varying parameter sizes RL algorithms