CoT-Valve: Length-Compressible Chain-of-Thought Tuning
Chain-of-Thought significantly enhances a model's reasoning capability, but
it also comes with a considerable increase in inference costs due to long
chains. With the observation that the reasoning path can be easily compressed
under easy tasks but struggle on hard tasks, we explore the feasibility of
elastically controlling the length of reasoning paths with only one model,
thereby reducing the inference overhead of reasoning models dynamically based
on task difficulty. We introduce a new tuning and inference strategy named
CoT-Valve, designed to allow models to generate reasoning chains of varying
lengths. To achieve this, we propose to identify a direction in the parameter
space that, when manipulated, can effectively control the length of generated
CoT. Moreover, we show that this property is valuable for compressing the
reasoning chain. We construct datasets with chains from long to short for the
same questions and explore two enhanced strategies for CoT-Valve: (1) a precise
length-compressible CoT tuning method, and (2) a progressive chain length
compression approach. Our experiments show that CoT-Valve successfully enables
controllability and compressibility of the chain and shows better performance
than the prompt-based control. We applied this method to QwQ-32B-Preview,
reducing reasoning chains on GSM8K from 741 to 225 tokens with a minor
performance drop (95.07% to 94.92%) and on AIME from 6827 to 4629 tokens, with
only one additional incorrect answer.
67aed173e6952709b47c0c5c
Xinyin Ma
admin_assigned
2025-02-14T12:50:50.673Z
64396ebc21221ac7411852b3
https://cdn-avatars.huggingface.co/v1/production/uploads/64396ebc21221ac7411852b3/SR0dC8N0bdj9tZFxYPpSf.jpeg
Xinyin Ma
user
horseee
67aed173e6952709b47c0c5d
Guangnian Wan
admin_assigned
2025-02-14T12:51:10.430Z
6627cccfded9b7936d5d1d21
https://cdn-avatars.huggingface.co/v1/production/uploads/6627cccfded9b7936d5d1d21/LKGr7EP7AirjmkbFZSn4o.jpeg
Guangnian Wan
user
bigglesworthnotcat
67aed173e6952709b47c0c5e
Runpeng Yu
admin_assigned
2025-02-14T12:51:16.707Z
635364b3c41f548fe39db945
/avatars/ad1916bbfabca0b6651c8eabacc5eba8.svg
Runpeng Yu
user
rp-yu
67aed173e6952709b47c0c5f
Gongfan Fang
admin_assigned
2025-02-14T12:51:22.345Z
646a1939c37ca1e12308fe81
/avatars/752e9d86018e7d33ad8bcd741203fd86.svg
Gongfan Fang
user
Vinnnf
67aed173e6952709b47c0c60
Xinchao Wang
CoT-Valve: Length-Compressible Chain-of-Thought Tuning
Chain-of-Thought significantly enhances a model's reasoning capability, but
it also comes with a considerable increase in inference costs due to long
chains. With the observation that the reasoning path can be easily compressed
under easy tasks but struggle on hard tasks, we explore the feasibility of
elastically controlling the length of reasoning paths with only one model,
thereby reducing the inference overhead of reasoning models dynamically based
on task difficulty. We introduce a new tuning and inference strategy named
CoT-Valve, designed to allow models to generate reasoning chains of varying
lengths. To achieve this, we propose to identify a direction in the parameter
space that, when manipulated, can effectively control the length of generated
CoT. Moreover, we show that this property is valuable for compressing the
reasoning chain. We construct datasets with chains from long to short for the
same questions and explore two enhanced strategies for CoT-Valve: (1) a precise
length-compressible CoT tuning method, and (2) a progressive chain length
compression approach. Our experiments show that CoT-Valve successfully enables
controllability and compressibility of the chain and shows better performance
than the prompt-based control. We applied this method to QwQ-32B-Preview,
reducing reasoning chains on GSM8K from 741 to 225 tokens with a minor
performance drop (95.07% to 94.92%) and on AIME from 6827 to 4629 tokens, with
only one additional incorrect answer.
A new strategy called CoT-Valve dynamically controls and compresses reasoning chains in models based on task difficulty, reducing inference costs with minimal performance impact.
Chain-of-Thought inference costs reasoning paths CoT-Valve parameter space length-compressible CoT tuning progressive chain length compression GSM8K AIME