Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic
  Thought Reward
Large language models (LLMs) exhibit remarkable problem-solving abilities,
but struggle with complex tasks due to static internal knowledge.
Retrieval-Augmented Generation (RAG) enhances access to external information,
yet remains limited in multi-hop reasoning and strategic search due to rigid
workflows. Recent advancements in agentic deep research empower LLMs to
autonomously reason, search, and synthesize information. However, current
approaches relying on outcome-based reinforcement learning (RL) face critical
issues such as conflicting gradients and reward sparsity, limiting performance
gains and training efficiency. To address these, we first propose Atomic
Thought, a novel LLM thinking paradigm that decomposes reasoning into
fine-grained functional units. These units are supervised by Reasoning Reward
Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained
guidance. Building on this, we propose Atom-Searcher, a novel RL framework for
agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher
uses a curriculum-inspired reward schedule, prioritizing process-level ATR
early and transitioning to outcome rewards, accelerating convergence on
effective reasoning paths. Experiments on seven benchmarks show consistent
improvements over the state-of-the-art. Key advantages include: (1)
Atom-Searcher scales computation at test-time. (2) Atomic Thought provides
supervision anchors for RRMs, bridging deep research tasks and RRMs. (3)
Atom-Searcher exhibits more interpretable, human-like reasoning patterns.
68a55dc46cf0bf898542ecf6
Yong Deng
claimed_verified
2025-08-20T08:51:11.768Z
63ea134a43d976de6e4e8ea7
https://cdn-avatars.huggingface.co/v1/production/uploads/63ea134a43d976de6e4e8ea7/OsyE7SYDW9foqsyWXtTiL.png
deng yong
user
dikw
68a55dc46cf0bf898542ecf7
Guoqing Wang
68a55dc46cf0bf898542ecf8
Zhenzhe Ying
68a55dc46cf0bf898542ecf9
Xiaofeng Wu
68a55dc46cf0bf898542ecfa
Jinzhen Lin
68a55dc46cf0bf898542ecfb
Wenwen Xiong
68a55dc46cf0bf898542ecfc
Yuqin Dai
68a55dc46cf0bf898542ecfd
Shuo Yang
68a55dc46cf0bf898542ecfe
Zhanwei Zhang
68a55dc46cf0bf898542ecff
Qiwen Wang
68a55dc46cf0bf898542ed00
Yang Qin
68a55dc46cf0bf898542ed01
Changhua Meng
Atom-Searcher: Enhancing Agentic Deep Research via Fine-Grained Atomic
  Thought Reward
Large language models (LLMs) exhibit remarkable problem-solving abilities,
but struggle with complex tasks due to static internal knowledge.
Retrieval-Augmented Generation (RAG) enhances access to external information,
yet remains limited in multi-hop reasoning and strategic search due to rigid
workflows. Recent advancements in agentic deep research empower LLMs to
autonomously reason, search, and synthesize information. However, current
approaches relying on outcome-based reinforcement learning (RL) face critical
issues such as conflicting gradients and reward sparsity, limiting performance
gains and training efficiency. To address these, we first propose Atomic
Thought, a novel LLM thinking paradigm that decomposes reasoning into
fine-grained functional units. These units are supervised by Reasoning Reward
Models (RRMs), which provide Atomic Thought Rewards (ATR) for fine-grained
guidance. Building on this, we propose Atom-Searcher, a novel RL framework for
agentic deep research that integrates Atomic Thought and ATR. Atom-Searcher
uses a curriculum-inspired reward schedule, prioritizing process-level ATR
early and transitioning to outcome rewards, accelerating convergence on
effective reasoning paths. Experiments on seven benchmarks show consistent
improvements over the state-of-the-art. Key advantages include: (1)
Atom-Searcher scales computation at test-time. (2) Atomic Thought provides
supervision anchors for RRMs, bridging deep research tasks and RRMs. (3)
Atom-Searcher exhibits more interpretable, human-like reasoning patterns.
Atom-Searcher, an RL framework integrating Atomic Thought and Reasoning Reward Models, enhances LLMs' multi-hop reasoning and strategic search capabilities, improving performance and interpretability.
Retrieval-Augmented Generation RAG agentic deep research reinforcement learning RL Atomic Thought Reasoning Reward Models RRMs Atomic Thought Rewards ATR curriculum-inspired reward schedule multi-hop reasoning strategic search interpretable reasoning