Thinking-Free Policy Initialization Makes Distilled Reasoning Models
  More Effective and Efficient Reasoners
Reinforcement Learning with Verifiable Reward (RLVR) effectively solves
complex tasks but demands extremely long context lengths during training,
leading to substantial computational costs. While multi-stage training can
partially mitigate this, starting with overly short contexts often causes
irreversible performance degradation, ultimately failing to reduce overall
training compute significantly. In this paper, we introduce
**T**hinking-**F**ree **P**olicy **I**nitialization (**TFPI**), a simple yet
effective adaptation to RLVR that bridges long Chain-of-Thought (CoT)
distillation and standard RLVR. TFPI employs a simple *ThinkFree* operation,
explicitly discarding the thinking content via a direct *</think>* append, to
reduce token usage during inference. Training with *ThinkFree*-adapted inputs
improves performance and lowers token consumption, even in the original
slow-thinking mode. Extensive experiments across various benchmarks have shown
that TFPI accelerates RL convergence, achieves a higher performance ceiling,
and yields more token-efficient reasoning models without specialized rewards or
complex training designs. With TFPI only, we train a 4B model to reach 89.0%
accuracy on AIME24 and 65.5% on LiveCodeBench using less than 4K H20 hours.
68dd2c3fe795697646c0a181
Xin Xu
claimed_verified
2025-10-02T03:08:34.526Z
64e2d169d2af12910d682130
https://cdn-avatars.huggingface.co/v1/production/uploads/64e2d169d2af12910d682130/VG8UdqJCJGc0K4G0P0XQP.jpeg
xuxin
user
xx18
68dd2c3fe795697646c0a182
Cliveb AI
68dd2c3fe795697646c0a183
Kai Yang
68dd2c3fe795697646c0a184
Tianhao Chen
68dd2c3fe795697646c0a185
Yang Wang
68dd2c3fe795697646c0a186
Saiyong Yang
68dd2c3fe795697646c0a187
Can Yang
Thinking-Free Policy Initialization Makes Distilled Reasoning Models
  More Effective and Efficient Reasoners
Reinforcement Learning with Verifiable Reward (RLVR) effectively solves
complex tasks but demands extremely long context lengths during training,
leading to substantial computational costs. While multi-stage training can
partially mitigate this, starting with overly short contexts often causes
irreversible performance degradation, ultimately failing to reduce overall
training compute significantly. In this paper, we introduce
**T**hinking-**F**ree **P**olicy **I**nitialization (**TFPI**), a simple yet
effective adaptation to RLVR that bridges long Chain-of-Thought (CoT)
distillation and standard RLVR. TFPI employs a simple *ThinkFree* operation,
explicitly discarding the thinking content via a direct *</think>* append, to
reduce token usage during inference. Training with *ThinkFree*-adapted inputs
improves performance and lowers token consumption, even in the original
slow-thinking mode. Extensive experiments across various benchmarks have shown
that TFPI accelerates RL convergence, achieves a higher performance ceiling,
and yields more token-efficient reasoning models without specialized rewards or
complex training designs. With TFPI only, we train a 4B model to reach 89.0%
accuracy on AIME24 and 65.5% on LiveCodeBench using less than 4K H20 hours.
TFPI, a simple adaptation to RLVR, improves performance and reduces token usage by discarding thinking content during training, accelerating RL convergence and achieving higher accuracy with less computational cost.
Reinforcement Learning with Verifiable Reward (RLVR) Chain-of-Thought (CoT) distillation ThinkFree operation token usage RL convergence performance ceiling token-efficient reasoning models AIME24 LiveCodeBench