Group Sequence Policy Optimization
This paper introduces Group Sequence Policy Optimization (GSPO), our stable,
efficient, and performant reinforcement learning algorithm for training large
language models. Unlike previous algorithms that adopt token-level importance
ratios, GSPO defines the importance ratio based on sequence likelihood and
performs sequence-level clipping, rewarding, and optimization. We demonstrate
that GSPO achieves superior training efficiency and performance compared to the
GRPO algorithm, notably stabilizes Mixture-of-Experts (MoE) RL training, and
has the potential for simplifying the design of RL infrastructure. These merits
of GSPO have contributed to the remarkable improvements in the latest Qwen3
models.
68831fe6846d2e78b7548ac8
Chujie Zheng
claimed_verified
2025-07-25T13:36:28.634Z
610b70452719facd4ea85e28
https://cdn-avatars.huggingface.co/v1/production/uploads/610b70452719facd4ea85e28/S7nMy7D0Rxq0VIVblhYDG.jpeg
Chujie Zheng
user
chujiezheng
68831fe6846d2e78b7548ac9
Shixuan Liu
claimed_verified
2025-07-29T07:10:41.029Z
64e72776e9fc9d0475ef5188
/avatars/d32b9d4e1da5486c3d5f9b04fa29d167.svg
Shixuan Liu
user
liusx
68831fe6846d2e78b7548aca
Mingze Li
68831fe6846d2e78b7548acb
Xiong-Hui Chen
claimed_verified
2025-07-25T13:36:26.841Z
63f30b870a16587ea970edfe
/avatars/b58ab2d8a85a6d83462c297de2714ce4.svg
Xiong-Hui Chen
user
xionghuichen
68831fe6846d2e78b7548acc
Bowen Yu
68831fe6846d2e78b7548acd
Chang Gao
68831fe6846d2e78b7548ace
Kai Dang
68831fe6846d2e78b7548acf
Yuqiong Liu
68831fe6846d2e78b7548ad0
Rui Men
68831fe6846d2e78b7548ad1
An Yang
claimed_verified
2025-07-28T08:46:14.463Z
62088594a5943c8a8fc94560
https://cdn-avatars.huggingface.co/v1/production/uploads/1644733028938-62088594a5943c8a8fc94560.png
An Yang
user
yangapku
68831fe6846d2e78b7548ad2
Jingren Zhou
68831fe6846d2e78b7548ad3
Junyang Lin
Group Sequence Policy Optimization
This paper introduces Group Sequence Policy Optimization (GSPO), our stable,
efficient, and performant reinforcement learning algorithm for training large
language models. Unlike previous algorithms that adopt token-level importance
ratios, GSPO defines the importance ratio based on sequence likelihood and
performs sequence-level clipping, rewarding, and optimization. We demonstrate
that GSPO achieves superior training efficiency and performance compared to the
GRPO algorithm, notably stabilizes Mixture-of-Experts (MoE) RL training, and
has the potential for simplifying the design of RL infrastructure. These merits
of GSPO have contributed to the remarkable improvements in the latest Qwen3
models.
Group Sequence Policy Optimization (GSPO) is a reinforcement learning algorithm that improves training efficiency and performance of large language models by using sequence-level importance ratios and operations.
Group Sequence Policy Optimization GSPO reinforcement learning large language models sequence likelihood sequence-level clipping rewarding optimization GRPO Mixture-of-Experts MoE RL Qwen3