GEM: A Gym for Agentic LLMs
The training paradigm for large language models (LLMs) is moving from static
datasets to experience-based learning, where agents acquire skills via
interacting with complex environments. To facilitate this transition we
introduce GEM (General Experience Maker), an open-source environment simulator
designed for the age of LLMs. Analogous to OpenAI-Gym for traditional
reinforcement learning (RL), GEM provides a standardized framework for the
environment-agent interface, including asynchronous vectorized execution for
high throughput, and flexible wrappers for easy extensibility. GEM also
features a diverse suite of environments, robust integrated tools, and
single-file example scripts demonstrating using GEM with five popular RL
training frameworks. Along with this, we also provide a set of baselines across
24 environments using REINFORCE with Return Batch Normalization (ReBN), which
-- unlike GRPO -- is compatible with the full RL setting of dense per-turn
rewards and offers better credit assignment. We further conduct apple-to-apple
benchmarking of PPO, GRPO and REINFORCE in both single- and multi-turn settings
using GEM to shed light on the algorithmic designs. Lastly, GEM also functions
as a convenient evaluation toolkit besides a training environment. We hope this
framework can help accelerate future agentic LLM research.
68ddde806024653e8a3ed0a9
Zichen Liu
claimed_verified
2025-10-02T03:07:51.006Z
65f5392c68b8e0cb3c9977a2
/avatars/aa64772475098e8a135c13072fde6744.svg
Zichen
user
lkevinzc
68ddde806024653e8a3ed0aa
Anya Sims
68ddde806024653e8a3ed0ab
Keyu Duan
claimed_verified
2025-10-02T13:55:22.843Z
6336a11331efcb5647ef32c8
https://cdn-avatars.huggingface.co/v1/production/uploads/6336a11331efcb5647ef32c8/mM0eC0mIV4W4W7-R-cleV.png
Keyu Duan
user
vermouthdky
68ddde806024653e8a3ed0ac
Changyu Chen
claimed_verified
2025-10-02T13:55:16.488Z
64e416dc54e18f390ef79ba4
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/5n01J00ZaVRrebsON8iYA.jpeg
Changyu Chen
user
Cameron-Chen
68ddde806024653e8a3ed0ad
Simon Yu
claimed_verified
2025-10-08T08:03:27.583Z
636681feaa6a4af6073ba73e
https://cdn-avatars.huggingface.co/v1/production/uploads/636681feaa6a4af6073ba73e/u_0moYNu6as9Sszp-ej95.png
Simon Yu
user
simonycl
68ddde806024653e8a3ed0ae
Xiangxin Zhou
claimed_verified
2025-10-02T13:55:20.132Z
66129c7b50350afe76757262
/avatars/a2f4fac076b9d658a0d904ed54960f6f.svg
Xiangxin Zhou
user
zhouxiangxin
68ddde806024653e8a3ed0af
Haotian Xu
68ddde806024653e8a3ed0b0
Shaopan Xiong
68ddde806024653e8a3ed0b1
Bo Liu
claimed_verified
2025-10-02T03:07:48.166Z
635e3a76106f984574c36409
https://cdn-avatars.huggingface.co/v1/production/uploads/1667120725800-635e3a76106f984574c36409.png
Bo Liu
user
Benjamin-eecs
68ddde806024653e8a3ed0b2
Chenmien Tan
68ddde806024653e8a3ed0b3
Chuen Yang Beh
admin_assigned
2025-10-17T17:27:53.546Z
654b692f910c8d5d07f889c4
/avatars/d0ba1dc453af32942cebd89d319f4493.svg
Beh Chuen Yang
user
N00bcak
68ddde806024653e8a3ed0b4
Weixun Wang
68ddde806024653e8a3ed0b5
Hao Zhu
68ddde806024653e8a3ed0b6
Weiyan Shi
68ddde806024653e8a3ed0b7
Diyi Yang
68ddde806024653e8a3ed0b8
Michael Shieh
68ddde806024653e8a3ed0b9
Yee Whye Teh
68ddde806024653e8a3ed0ba
Wee Sun Lee
68ddde806024653e8a3ed0bb
Min Lin
GEM: A Gym for Agentic LLMs
The training paradigm for large language models (LLMs) is moving from static
datasets to experience-based learning, where agents acquire skills via
interacting with complex environments. To facilitate this transition we
introduce GEM (General Experience Maker), an open-source environment simulator
designed for the age of LLMs. Analogous to OpenAI-Gym for traditional
reinforcement learning (RL), GEM provides a standardized framework for the
environment-agent interface, including asynchronous vectorized execution for
high throughput, and flexible wrappers for easy extensibility. GEM also
features a diverse suite of environments, robust integrated tools, and
single-file example scripts demonstrating using GEM with five popular RL
training frameworks. Along with this, we also provide a set of baselines across
24 environments using REINFORCE with Return Batch Normalization (ReBN), which
-- unlike GRPO -- is compatible with the full RL setting of dense per-turn
rewards and offers better credit assignment. We further conduct apple-to-apple
benchmarking of PPO, GRPO and REINFORCE in both single- and multi-turn settings
using GEM to shed light on the algorithmic designs. Lastly, GEM also functions
as a convenient evaluation toolkit besides a training environment. We hope this
framework can help accelerate future agentic LLM research.
GEM, an open-source environment simulator, facilitates experience-based learning for large language models by providing a standardized framework and diverse environments for training and benchmarking reinforcement learning algorithms.
large language models experience-based learning GEM environment simulator OpenAI-Gym asynchronous vectorized execution flexible wrappers REINFORCE Return Batch Normalization ReBN GRPO PPO benchmarking agentic LLM research