AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making
  through Multi-Turn Reinforcement Learning
Developing autonomous LLM agents capable of making a series of intelligent
decisions to solve complex, real-world tasks is a fast-evolving frontier. Like
human cognitive development, agents are expected to acquire knowledge and
skills through exploration and interaction with the environment. Despite
advances, the community still lacks a unified, interactive reinforcement
learning (RL) framework that can effectively train such agents from scratch --
without relying on supervised fine-tuning (SFT) -- across diverse and realistic
environments. To bridge this gap, we introduce AgentGym-RL, a new framework to
train LLM agents for multi-turn interactive decision-making through RL. The
framework features a modular and decoupled architecture, ensuring high
flexibility and extensibility. It encompasses a wide variety of real-world
scenarios, and supports mainstream RL algorithms. Furthermore, we propose
ScalingInter-RL, a training approach designed for exploration-exploitation
balance and stable RL optimization. In early stages, it emphasizes exploitation
by restricting the number of interactions, and gradually shifts towards
exploration with larger horizons to encourage diverse problem-solving
strategies. In this way, the agent develops more diverse behaviors and is less
prone to collapse under long horizons. We perform extensive experiments to
validate the stability and effectiveness of both the AgentGym-RL framework and
the ScalingInter-RL approach. Our agents match or surpass commercial models on
27 tasks across diverse environments. We offer key insights and will
open-source the complete AgentGym-RL framework -- including code and datasets
-- to empower the research community in developing the next generation of
intelligent agents.
68c2329529b8ec9932cd090a
Zhiheng Xi
68c2329529b8ec9932cd090b
Jixuan Huang
68c2329529b8ec9932cd090c
Chenyang Liao
68c2329529b8ec9932cd090d
Baodai Huang
68c2329529b8ec9932cd090e
Honglin Guo
68c2329529b8ec9932cd090f
Jiaqi Liu
68c2329529b8ec9932cd0910
Rui Zheng
68c2329529b8ec9932cd0911
Junjie Ye
claimed_verified
2025-09-12T16:13:38.007Z
66384be673c2c55f2ded89fa
/avatars/1d8721074f0f51fab405f81474f2035f.svg
Junjie Ye
user
Junjie-Ye
68c2329529b8ec9932cd0912
Jiazheng Zhang
68c2329529b8ec9932cd0913
Wenxiang Chen
68c2329529b8ec9932cd0914
Wei He
68c2329529b8ec9932cd0915
Yiwen Ding
68c2329529b8ec9932cd0916
Guanyu Li
68c2329529b8ec9932cd0917
Zehui Chen
68c2329529b8ec9932cd0918
Zhengyin Du
68c2329529b8ec9932cd0919
Xuesong Yao
68c2329529b8ec9932cd091a
Yufei Xu
68c2329529b8ec9932cd091b
Jiecao Chen
68c2329529b8ec9932cd091c
Tao Gui
68c2329529b8ec9932cd091d
Zuxuan Wu
68c2329529b8ec9932cd091e
Qi Zhang
68c2329529b8ec9932cd091f
Xuanjing Huang
68c2329529b8ec9932cd0920
Yu-Gang Jiang
AgentGym-RL: Training LLM Agents for Long-Horizon Decision Making
  through Multi-Turn Reinforcement Learning
Developing autonomous LLM agents capable of making a series of intelligent
decisions to solve complex, real-world tasks is a fast-evolving frontier. Like
human cognitive development, agents are expected to acquire knowledge and
skills through exploration and interaction with the environment. Despite
advances, the community still lacks a unified, interactive reinforcement
learning (RL) framework that can effectively train such agents from scratch --
without relying on supervised fine-tuning (SFT) -- across diverse and realistic
environments. To bridge this gap, we introduce AgentGym-RL, a new framework to
train LLM agents for multi-turn interactive decision-making through RL. The
framework features a modular and decoupled architecture, ensuring high
flexibility and extensibility. It encompasses a wide variety of real-world
scenarios, and supports mainstream RL algorithms. Furthermore, we propose
ScalingInter-RL, a training approach designed for exploration-exploitation
balance and stable RL optimization. In early stages, it emphasizes exploitation
by restricting the number of interactions, and gradually shifts towards
exploration with larger horizons to encourage diverse problem-solving
strategies. In this way, the agent develops more diverse behaviors and is less
prone to collapse under long horizons. We perform extensive experiments to
validate the stability and effectiveness of both the AgentGym-RL framework and
the ScalingInter-RL approach. Our agents match or surpass commercial models on
27 tasks across diverse environments. We offer key insights and will
open-source the complete AgentGym-RL framework -- including code and datasets
-- to empower the research community in developing the next generation of
intelligent agents.
AgentGym-RL is a modular RL framework for training LLM agents in diverse environments without supervised fine-tuning, featuring ScalingInter-RL for balanced exploration-exploitation.
reinforcement learning RL modular architecture decoupled architecture exploration-exploitation balance ScalingInter-RL LLM agents multi-turn decision-making diverse environments commercial models