AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced
  Self-Play
Search-augmented LLMs often struggle with complex reasoning tasks due to
ineffective multi-hop retrieval and limited reasoning ability. We propose
AceSearcher, a cooperative self-play framework that trains a single large
language model (LLM) to alternate between two roles: a decomposer that breaks
down complex queries and a solver that integrates retrieved contexts for answer
generation. AceSearcher couples supervised fine-tuning on a diverse mixture of
search, reasoning, and decomposition tasks with reinforcement fine-tuning
optimized for final answer accuracy, eliminating the need for intermediate
annotations. Extensive experiments on three reasoning-intensive tasks across 10
datasets show that AceSearcher outperforms state-of-the-art baselines,
achieving an average exact match improvement of 7.6%. Remarkably, on
document-level finance reasoning tasks, AceSearcher-32B matches the performance
of the DeepSeek-V3 model using less than 5% of its parameters. Even at smaller
scales (1.5B and 8B), AceSearcher often surpasses existing search-augmented
LLMs with up to 9x more parameters, highlighting its exceptional efficiency and
effectiveness in tackling complex reasoning tasks. Our code will be published
at https://github.com/ritaranx/AceSearcher and
https://huggingface.co/AceSearcher.
68db4081d2bf1f4b15ec72dc
Ran Xu
claimed_verified
2025-10-13T10:10:50.036Z
6621dd7623518bf2898788da
/avatars/837aca71471a19ea80ae55d619781c5a.svg
Ran Xu
user
ritaranx
68db4081d2bf1f4b15ec72dd
Yuchen Zhuang
68db4081d2bf1f4b15ec72de
Zihan Dong
68db4081d2bf1f4b15ec72df
Jonathan Wang
68db4081d2bf1f4b15ec72e0
Yue Yu
68db4081d2bf1f4b15ec72e1
Joyce C. Ho
68db4081d2bf1f4b15ec72e2
Linjun Zhang
68db4081d2bf1f4b15ec72e3
Haoyu Wang
68db4081d2bf1f4b15ec72e4
Wenqi Shi
68db4081d2bf1f4b15ec72e5
Carl Yang
AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced
  Self-Play
Search-augmented LLMs often struggle with complex reasoning tasks due to
ineffective multi-hop retrieval and limited reasoning ability. We propose
AceSearcher, a cooperative self-play framework that trains a single large
language model (LLM) to alternate between two roles: a decomposer that breaks
down complex queries and a solver that integrates retrieved contexts for answer
generation. AceSearcher couples supervised fine-tuning on a diverse mixture of
search, reasoning, and decomposition tasks with reinforcement fine-tuning
optimized for final answer accuracy, eliminating the need for intermediate
annotations. Extensive experiments on three reasoning-intensive tasks across 10
datasets show that AceSearcher outperforms state-of-the-art baselines,
achieving an average exact match improvement of 7.6%. Remarkably, on
document-level finance reasoning tasks, AceSearcher-32B matches the performance
of the DeepSeek-V3 model using less than 5% of its parameters. Even at smaller
scales (1.5B and 8B), AceSearcher often surpasses existing search-augmented
LLMs with up to 9x more parameters, highlighting its exceptional efficiency and
effectiveness in tackling complex reasoning tasks. Our code will be published
at https://github.com/ritaranx/AceSearcher and
https://huggingface.co/AceSearcher.
AceSearcher, a cooperative self-play framework, enhances a large language model's reasoning ability by alternating between decomposing queries and solving them, outperforming state-of-the-art models with fewer parameters.
large language model decomposer solver supervised fine-tuning reinforcement fine-tuning exact match improvement document-level finance reasoning parameter efficiency