FastCuRL: Curriculum Reinforcement Learning with Progressive Context
  Extension for Efficient Training R1-like Reasoning Models
In this paper, we propose \textsc{FastCuRL}, a simple yet efficient
Curriculum Reinforcement Learning approach with
context window extending strategy to accelerate the reinforcement learning
training efficiency for R1-like reasoning models while enhancing their
performance in tackling complex reasoning tasks with long chain-of-thought
rationales, particularly with a 1.5B parameter language model.
\textsc{FastCuRL} consists of two main procedures: length-aware
training data segmentation and context window extension training. Specifically,
the former first splits the original training data into three different levels
by the input prompt length, and then the latter leverages segmented training
datasets with a progressively increasing context window length to train the
reasoning model. Experimental results demonstrate that
\textsc{FastCuRL}-1.5B-Preview surpasses DeepScaleR-1.5B-Preview
across all five datasets (including MATH 500, AIME 2024, AMC 2023, Minerva
Math, and OlympiadBench) while only utilizing 50\% of training steps.
Furthermore, all training stages for FastCuRL-1.5B-Preview are completed using
just a single node with 8 GPUs.
67e0bfcc8fb92b0edaa78dc0
Mingyang Song
claimed_verified
2025-04-28T12:56:34.911Z
617051728db4a760d912d81f
https://cdn-avatars.huggingface.co/v1/production/uploads/617051728db4a760d912d81f/4dKG176GPk2pPRK7ctaT-.jpeg
Mingyang Song
user
Nickyang
67e0bfcc8fb92b0edaa78dc1
Mao Zheng
67e0bfcc8fb92b0edaa78dc2
Zheng Li
67e0bfcc8fb92b0edaa78dc3
Wenjie Yang
67e0bfcc8fb92b0edaa78dc4
Xuan Luo
67e0bfcc8fb92b0edaa78dc5
Yue Pan
67e0bfcc8fb92b0edaa78dc6
Feng Zhang
FastCuRL: Curriculum Reinforcement Learning with Progressive Context
  Extension for Efficient Training R1-like Reasoning Models
In this paper, we propose \textsc{FastCuRL}, a simple yet efficient
Curriculum Reinforcement Learning approach with
context window extending strategy to accelerate the reinforcement learning
training efficiency for R1-like reasoning models while enhancing their
performance in tackling complex reasoning tasks with long chain-of-thought
rationales, particularly with a 1.5B parameter language model.
\textsc{FastCuRL} consists of two main procedures: length-aware
training data segmentation and context window extension training. Specifically,
the former first splits the original training data into three different levels
by the input prompt length, and then the latter leverages segmented training
datasets with a progressively increasing context window length to train the
reasoning model. Experimental results demonstrate that
\textsc{FastCuRL}-1.5B-Preview surpasses DeepScaleR-1.5B-Preview
across all five datasets (including MATH 500, AIME 2024, AMC 2023, Minerva
Math, and OlympiadBench) while only utilizing 50\% of training steps.
Furthermore, all training stages for FastCuRL-1.5B-Preview are completed using
just a single node with 8 GPUs.
FastCuRL is a curriculum reinforcement learning method that accelerates training and improves performance on complex reasoning tasks using segmented data and progressively increased context windows.
Curriculum Reinforcement Learning context window length-aware training reasoning models chain-of-thought language model MATH 500 AIME 2024 AMC 2023 Minerva Math OlympiadBench