O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple
  Distillation, Big Progress or Bitter Lesson?
This paper presents a critical examination of current approaches to
replicating OpenAI's O1 model capabilities, with particular focus on the
widespread but often undisclosed use of knowledge distillation techniques.
While our previous work explored the fundamental technical path to O1
replication, this study reveals how simple distillation from O1's API, combined
with supervised fine-tuning, can achieve superior performance on complex
mathematical reasoning tasks. Through extensive experiments, we show that a
base model fine-tuned on simply tens of thousands of samples O1-distilled
long-thought chains outperforms O1-preview on the American Invitational
Mathematics Examination (AIME) with minimal technical complexity. Moreover, our
investigation extends beyond mathematical reasoning to explore the
generalization capabilities of O1-distilled models across diverse tasks:
hallucination, safety and open-domain QA. Notably, despite training only on
mathematical problem-solving data, our models demonstrated strong
generalization to open-ended QA tasks and became significantly less susceptible
to sycophancy after fine-tuning. We deliberately make this finding public to
promote transparency in AI research and to challenge the current trend of
obscured technical claims in the field. Our work includes: (1) A detailed
technical exposition of the distillation process and its effectiveness, (2) A
comprehensive benchmark framework for evaluating and categorizing O1
replication attempts based on their technical transparency and reproducibility,
(3) A critical discussion of the limitations and potential risks of
over-relying on distillation approaches, our analysis culminates in a crucial
bitter lesson: while the pursuit of more capable AI systems is important, the
development of researchers grounded in first-principles thinking is paramount.
6745580279d6f3a9da792790
Zhen Huang
admin_assigned
2024-11-26T22:43:14.316Z
643581a4f3b08e267d990499
https://cdn-avatars.huggingface.co/v1/production/uploads/643581a4f3b08e267d990499/KRhB-48W4IPuB0bX16Ahj.png
Zhen Huang
user
ZhenHuang
6745580279d6f3a9da792791
Haoyang Zou
admin_assigned
2024-11-26T22:43:08.617Z
65011b18af1e4021439af578
/avatars/ae1632c800ceeb43aa223e432f70bc6d.svg
Haoyang Zou
user
alanyoung058
6745580279d6f3a9da792792
Xuefeng Li
admin_assigned
2024-11-26T22:43:03.086Z
64e3562342d8e2c1c69f64b3
/avatars/1ccf22d60deec213bbe069d30811efbe.svg
xuefengli
user
xuefengli
6745580279d6f3a9da792793
Yixiu Liu
6745580279d6f3a9da792794
Yuxiang Zheng
claimed_verified
2025-05-13T08:13:57.942Z
64dc2b8dff83b3386a5a897c
https://cdn-avatars.huggingface.co/v1/production/uploads/64dc2b8dff83b3386a5a897c/nxYnIXWmJRDuyw_EmqEAd.png
Yuxiang Zheng
user
catchiz
6745580279d6f3a9da792795
Ethan Chern
admin_assigned
2024-11-26T22:42:37.230Z
64bb5f9d8e051085bace4d1e
/avatars/15ccbb78c6131dfe46b7a9d8e7d1a31f.svg
Ethan Chern
user
ethanchern
6745580279d6f3a9da792796
Shijie Xia
admin_assigned
2024-11-26T22:42:31.279Z
65900d4ff5a209eeac08b463
https://cdn-avatars.huggingface.co/v1/production/uploads/65900d4ff5a209eeac08b463/PJNNBRJIk1qR24oaRLTex.jpeg
shijie xia
user
seven-cat
6745580279d6f3a9da792797
Yiwei Qin
6745580279d6f3a9da792798
Weizhe Yuan
admin_assigned
2024-11-26T22:42:08.375Z
6172aa78c8e66e2aa84c06b8
https://cdn-avatars.huggingface.co/v1/production/uploads/1634904686602-noauth.png
Weizhe Yuan
user
weizhey
6745580279d6f3a9da792799
Pengfei Liu
admin_assigned
2024-11-26T22:42:01.710Z
6144a0c4ff1146bbd84d9865
https://cdn-avatars.huggingface.co/v1/production/uploads/1661715958139-6144a0c4ff1146bbd84d9865.png
Pengfei Liu
user
Pengfei
O1 Replication Journey -- Part 2: Surpassing O1-preview through Simple
  Distillation, Big Progress or Bitter Lesson?
This paper presents a critical examination of current approaches to
replicating OpenAI's O1 model capabilities, with particular focus on the
widespread but often undisclosed use of knowledge distillation techniques.
While our previous work explored the fundamental technical path to O1
replication, this study reveals how simple distillation from O1's API, combined
with supervised fine-tuning, can achieve superior performance on complex
mathematical reasoning tasks. Through extensive experiments, we show that a
base model fine-tuned on simply tens of thousands of samples O1-distilled
long-thought chains outperforms O1-preview on the American Invitational
Mathematics Examination (AIME) with minimal technical complexity. Moreover, our
investigation extends beyond mathematical reasoning to explore the
generalization capabilities of O1-distilled models across diverse tasks:
hallucination, safety and open-domain QA. Notably, despite training only on
mathematical problem-solving data, our models demonstrated strong
generalization to open-ended QA tasks and became significantly less susceptible
to sycophancy after fine-tuning. We deliberately make this finding public to
promote transparency in AI research and to challenge the current trend of
obscured technical claims in the field. Our work includes: (1) A detailed
technical exposition of the distillation process and its effectiveness, (2) A
comprehensive benchmark framework for evaluating and categorizing O1
replication attempts based on their technical transparency and reproducibility,
(3) A critical discussion of the limitations and potential risks of
over-relying on distillation approaches, our analysis culminates in a crucial
bitter lesson: while the pursuit of more capable AI systems is important, the
development of researchers grounded in first-principles thinking is paramount.
Knowledge distillation from O1 model combined with fine-tuning achieves superior performance on complex mathematical reasoning and generalizes to open-ended QA tasks, promoting transparency in AI research.
knowledge distillation O1 model fine-tuning American Invitational Mathematics Examination (AIME) hallucination safety open-domain QA sycophancy