One Missing Piece for Open-Source Reasoning Models: A Dataset to
  Mitigate Cold-Starting Short CoT LLMs in RL
With the release of R1, a publicly available large reasoning model (LRM),
researchers commonly train new LRMs by training language models on R1's long
chain-of-thought (CoT) inferences. While prior works show that LRMs'
capabilities can be reproduced through direct distillation, the continued
reliance on the existing models (e.g., R1) remains a critical limitation in
advancing the field. As a first step toward independent LRM development, this
paper explores the possibility of constructing a long CoT dataset with LLMs
that are not trained for inference-time scaling. To this end, we present the
Long CoT Collection, a dataset of 100K CoT rationales annotated using existing
short CoT LLMs. We develop a pipeline that induces o1's novel reasoning
strategies into short CoT LLMs, enabling them to think longer and introducing
controllability over the thought budget to better manage the overthinking
problem. Our extensive analyses validate that our dataset achieves quality
comparable to--or slightly below--R1. Furthermore, our experiments demonstrate
that training on our dataset not only strengthens general reasoning skills, but
also provides a strong foundation for reinforcement learning--models
initialized on our data achieve 2-3x larger gains with RLVR.
683fbc730ffa93c1611d513b
Hyungjoo Chae
claimed_verified
2025-06-04T08:55:14.169Z
64c8f4cec547ed5243ebd0a8
https://cdn-avatars.huggingface.co/v1/production/uploads/64c8f4cec547ed5243ebd0a8/MiOH5YbMg8Gh9KYlQsLmX.jpeg
Hyungjoo Chae
user
hyungjoochae
683fbc730ffa93c1611d513c
Dongjin Kang
683fbc730ffa93c1611d513d
Jihyuk Kim
683fbc730ffa93c1611d513e
Beong-woo Kwak
683fbc730ffa93c1611d513f
Sunghyun Park
683fbc730ffa93c1611d5140
Haeju Park
683fbc730ffa93c1611d5141
Jinyoung Yeo
683fbc730ffa93c1611d5142
Moontae Lee
683fbc730ffa93c1611d5143
Kyungjae Lee
One Missing Piece for Open-Source Reasoning Models: A Dataset to
  Mitigate Cold-Starting Short CoT LLMs in RL
With the release of R1, a publicly available large reasoning model (LRM),
researchers commonly train new LRMs by training language models on R1's long
chain-of-thought (CoT) inferences. While prior works show that LRMs'
capabilities can be reproduced through direct distillation, the continued
reliance on the existing models (e.g., R1) remains a critical limitation in
advancing the field. As a first step toward independent LRM development, this
paper explores the possibility of constructing a long CoT dataset with LLMs
that are not trained for inference-time scaling. To this end, we present the
Long CoT Collection, a dataset of 100K CoT rationales annotated using existing
short CoT LLMs. We develop a pipeline that induces o1's novel reasoning
strategies into short CoT LLMs, enabling them to think longer and introducing
controllability over the thought budget to better manage the overthinking
problem. Our extensive analyses validate that our dataset achieves quality
comparable to--or slightly below--R1. Furthermore, our experiments demonstrate
that training on our dataset not only strengthens general reasoning skills, but
also provides a strong foundation for reinforcement learning--models
initialized on our data achieve 2-3x larger gains with RLVR.
The Long CoT Collection dataset, generated by short CoT LLMs, enhances general reasoning skills and provides a strong foundation for reinforcement learning, achieving quality comparable to R1.
long chain-of-thought CoT inferences LRMs direct distillation inference-time scaling CoT rationales short CoT LLMs reasoning strategies thought budget overthinking reinforcement learning RLVR