Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language
  Models in Math
Chain-of-Thought (CoT) significantly enhances formal reasoning capabilities
in Large Language Models (LLMs) by training them to explicitly generate
intermediate reasoning steps. While LLMs readily benefit from such techniques,
improving reasoning in Small Language Models (SLMs) remains challenging due to
their limited model capacity. Recent work by Deepseek-R1 demonstrates that
distillation from LLM-generated synthetic data can substantially improve the
reasoning ability of SLM. However, the detailed modeling recipe is not
disclosed. In this work, we present a systematic training recipe for SLMs that
consists of four steps: (1) large-scale mid-training on diverse distilled
long-CoT data, (2) supervised fine-tuning on high-quality long-CoT data, (3)
Rollout DPO leveraging a carefully curated preference dataset, and (4)
Reinforcement Learning (RL) with Verifiable Reward. We apply our method on
Phi-4-Mini, a compact 3.8B-parameter model. The resulting Phi-4-Mini-Reasoning
model exceeds, on math reasoning tasks, much larger reasoning models, e.g.,
outperforming DeepSeek-R1-Distill-Qwen-7B by 3.2 points and
DeepSeek-R1-Distill-Llama-8B by 7.7 points on Math-500. Our results validate
that a carefully designed training recipe, with large-scale high-quality CoT
data, is effective to unlock strong reasoning capabilities even in
resource-constrained small models.
6812d62ae74b39182bd17c9c
Haoran Xu
admin_assigned
2025-05-05T07:53:11.914Z
61384b860317b0a5c10877d3
https://cdn-avatars.huggingface.co/v1/production/uploads/1631080954171-61384b860317b0a5c10877d3.jpeg
Haoran Xu
user
haoranxu
6812d62ae74b39182bd17c9d
Baolin Peng
admin_assigned
2025-05-05T07:53:31.380Z
61942296d5c2ba6daa290357
/avatars/594021cc183c4922d48b46f43772a062.svg
Baolin Peng
user
Baolin
6812d62ae74b39182bd17c9e
Hany Awadalla
6812d62ae74b39182bd17c9f
Dongdong Chen
admin_assigned
2025-05-05T07:53:43.028Z
6567651c6fcc82e5e8c36d4d
/avatars/ba3cc037a7688c4f8d967fc6043e540d.svg
Dongdong Chen
user
dongdongchen
6812d62ae74b39182bd17ca0
Yen-Chun Chen
6812d62ae74b39182bd17ca1
Mei Gao
6812d62ae74b39182bd17ca2
Young Jin Kim
claimed_verified
2025-05-02T06:35:02.393Z
63f5173bb51da4d61da6c038
/avatars/0ee530cf80476aa3985c4d591cd384a1.svg
Young Jin Kim
user
ykim362
6812d62ae74b39182bd17ca3
Yunsheng Li
admin_assigned
2025-05-05T07:54:15.501Z
65b9a5a498bd7a573bb98408
/avatars/efab4e24c36db6a5d50263677308cb8a.svg
Yunsheng Li
user
liyunsheng13
6812d62ae74b39182bd17ca4
Liliang Ren
admin_assigned
2025-05-05T07:54:21.232Z
63815eff4761ddfa00903762
https://cdn-avatars.huggingface.co/v1/production/uploads/63815eff4761ddfa00903762/YlkfX3sTZRQcoapctXJeJ.jpeg
Liliang Ren
user
renll
6812d62ae74b39182bd17ca5
Yelong Shen
admin_assigned
2025-05-05T07:54:30.183Z
6454c337a13edf669cd5d8ea
/avatars/a383a0dda7c2ef6a0d6c3c64651f42ff.svg
Yelong Shen
user
uuu6
6812d62ae74b39182bd17ca6
Shuohang Wang
admin_assigned
2025-05-05T07:54:38.929Z
6463b2247572c66a8e625a57
/avatars/7722fb5649d42d966ce1e478946d5f8f.svg
Wang
user
Shuohang
6812d62ae74b39182bd17ca7
Weijian Xu
admin_assigned
2025-05-05T07:54:44.400Z
6398f4b32c20654083f36cde
/avatars/4591f514483890997c55e9e6d60bbb0f.svg
Weijian Xu
user
xwjabc
6812d62ae74b39182bd17ca8
Jianfeng Gao
admin_assigned
2025-05-05T07:54:50.324Z
641904caf9d6f1d772ec7af7
/avatars/4a63eac71eb30f70b1a0e9d4708f26c1.svg
Jianfeng Gao
user
wyngjf
6812d62ae74b39182bd17ca9
Weizhu Chen
admin_assigned
2025-05-05T07:54:56.808Z
64da876370446182be5b608d
/avatars/e412fdc71404ecdf638e416846e3ebfb.svg
Weizhu Chen
user
chenweizhu
Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language
  Models in Math
Chain-of-Thought (CoT) significantly enhances formal reasoning capabilities
in Large Language Models (LLMs) by training them to explicitly generate
intermediate reasoning steps. While LLMs readily benefit from such techniques,
improving reasoning in Small Language Models (SLMs) remains challenging due to
their limited model capacity. Recent work by Deepseek-R1 demonstrates that
distillation from LLM-generated synthetic data can substantially improve the
reasoning ability of SLM. However, the detailed modeling recipe is not
disclosed. In this work, we present a systematic training recipe for SLMs that
consists of four steps: (1) large-scale mid-training on diverse distilled
long-CoT data, (2) supervised fine-tuning on high-quality long-CoT data, (3)
Rollout DPO leveraging a carefully curated preference dataset, and (4)
Reinforcement Learning (RL) with Verifiable Reward. We apply our method on
Phi-4-Mini, a compact 3.8B-parameter model. The resulting Phi-4-Mini-Reasoning
model exceeds, on math reasoning tasks, much larger reasoning models, e.g.,
outperforming DeepSeek-R1-Distill-Qwen-7B by 3.2 points and
DeepSeek-R1-Distill-Llama-8B by 7.7 points on Math-500. Our results validate
that a carefully designed training recipe, with large-scale high-quality CoT
data, is effective to unlock strong reasoning capabilities even in
resource-constrained small models.
A structured training method with high-quality Chain-of-Thought data improves reasoning abilities in small language models.
Chain-of-Thought Large Language Models Small Language Models distillation large-scale mid-training supervised fine-tuning Rollout DPO Reinforcement Learning Verifiable Reward math reasoning tasks