SFR-DeepResearch: Towards Effective Reinforcement Learning for
  Autonomously Reasoning Single Agents
Equipping large language models (LLMs) with complex, interleaved reasoning
and tool-use capabilities has become a key focus in agentic AI research,
especially with recent advances in reasoning-oriented (``thinking'') models.
Such capabilities are key to unlocking a number of important applications. One
such application is Deep Research (DR), which requires extensive search and
reasoning over many sources. Our work in this paper focuses on the development
of native Autonomous Single-Agent models for DR featuring minimal web crawling
and Python tool integration. Unlike multi-agent systems, where agents take up
pre-defined roles and are told what to do at each step in a static workflow, an
autonomous single-agent determines its next action dynamically based on
context, without manual directive. While prior work has proposed training
recipes for base or instruction-tuned LLMs, we focus on continual reinforcement
learning (RL) of reasoning-optimized models to further enhance agentic skills
while preserving reasoning ability. Towards this end, we propose a simple RL
recipe with entirely synthetic data, which we apply to various open-source
LLMs. Our best variant SFR-DR-20B achieves up to 28.7% on Humanity's Last Exam
benchmark. In addition, we conduct key analysis experiments to provide more
insights into our methodologies.
68c0b7f73912ed54cf5431b8
Xuan-Phi Nguyen
68c0b7f73912ed54cf5431b9
Shrey Pandit
claimed_verified
2025-09-12T16:18:55.038Z
648749094dea003c6dae810f
https://cdn-avatars.huggingface.co/v1/production/uploads/648749094dea003c6dae810f/gHUHSBt1zrt8wjO1YwTNu.jpeg
Shrey Pandit
user
SP2001
68c0b7f73912ed54cf5431ba
Revanth Gangi Reddy
68c0b7f73912ed54cf5431bb
Austin Xu
claimed_verified
2025-09-16T09:43:31.295Z
6668e86dc4ef4175fb18d250
/avatars/d34925685799a1218017f85ce4d44e6e.svg
Austin Xu
user
austinxu87
68c0b7f73912ed54cf5431bc
Silvio Savarese
68c0b7f73912ed54cf5431bd
Caiming Xiong
68c0b7f73912ed54cf5431be
Shafiq Joty
SFR-DeepResearch: Towards Effective Reinforcement Learning for
  Autonomously Reasoning Single Agents
Equipping large language models (LLMs) with complex, interleaved reasoning
and tool-use capabilities has become a key focus in agentic AI research,
especially with recent advances in reasoning-oriented (``thinking'') models.
Such capabilities are key to unlocking a number of important applications. One
such application is Deep Research (DR), which requires extensive search and
reasoning over many sources. Our work in this paper focuses on the development
of native Autonomous Single-Agent models for DR featuring minimal web crawling
and Python tool integration. Unlike multi-agent systems, where agents take up
pre-defined roles and are told what to do at each step in a static workflow, an
autonomous single-agent determines its next action dynamically based on
context, without manual directive. While prior work has proposed training
recipes for base or instruction-tuned LLMs, we focus on continual reinforcement
learning (RL) of reasoning-optimized models to further enhance agentic skills
while preserving reasoning ability. Towards this end, we propose a simple RL
recipe with entirely synthetic data, which we apply to various open-source
LLMs. Our best variant SFR-DR-20B achieves up to 28.7% on Humanity's Last Exam
benchmark. In addition, we conduct key analysis experiments to provide more
insights into our methodologies.
Continual reinforcement learning enhances autonomous single-agent models for deep research tasks, improving agentic skills and reasoning ability using synthetic data.
large language models reasoning-oriented models deep research autonomous single-agent models reinforcement learning reasoning-optimized models Humanity's Last Exam benchmark