BOLT: Bootstrap Long Chain-of-Thought in Language Models without
  Distillation
Large language models (LLMs), such as o1 from OpenAI, have demonstrated
remarkable reasoning capabilities. o1 generates a long chain-of-thought
(LongCoT) before answering a question. LongCoT allows LLMs to analyze problems,
devise plans, reflect, and backtrack effectively. These actions empower LLM to
solve complex problems. After the release of o1, many teams have attempted to
replicate its LongCoT and reasoning capabilities. In terms of methods, they
primarily rely on knowledge distillation with data from existing models with
LongCoT capacities (e.g., OpenAI-o1, Qwen-QwQ, DeepSeek-R1-Preview), leaving
significant uncertainties on systematically developing such reasoning
abilities. In terms of data domains, these works focus narrowly on math while a
few others include coding, limiting their generalizability. This paper
introduces a novel approach to enable LLM's LongCoT capacity without
distillation from o1-like models or expensive human annotations, where we
bootstrap LongCoT (BOLT) from a standard instruct model. BOLT involves three
stages: 1) LongCoT data bootstrapping with in-context learning on a standard
instruct model; 2) LongCoT supervised finetuning; 3) online training to further
refine LongCoT capacities. In BOLT, only a few in-context examples need to be
constructed during the bootstrapping stage; in our experiments, we created 10
examples, demonstrating the feasibility of this approach. We use
Llama-3.1-70B-Instruct to bootstrap LongCoT and apply our method to various
model scales (7B, 8B, 70B). We achieve impressive performance on a variety of
benchmarks, Arena-Hard, MT-Bench, WildBench, ZebraLogic, MATH500, which
evaluate diverse task-solving and reasoning capabilities.
67a5880c886a1e223b1d57ec
Bo Pang
admin_assigned
2025-02-07T10:46:17.484Z
63e08acbf351dc0745749d56
/avatars/8e2d5ce9db5bd8008ac2ad80f6025553.svg
Bo Pang
user
bpucla
67a5880c886a1e223b1d57ed
Hanze Dong
admin_assigned
2025-02-07T10:46:02.107Z
63a3ff69f91ad3ea5703841d
/avatars/69227c4bce01d33747c1377b6f9672db.svg
Hanze Dong
user
hendrydong
67a5880c886a1e223b1d57ee
Jiacheng Xu
admin_assigned
2025-02-07T10:45:54.892Z
631983d5cb116eab31df5821
/avatars/f7b01604b70ab0a7bfda66b37bfd01d1.svg
Jiacheng Xu
user
jcxu
67a5880c886a1e223b1d57ef
Silvio Savarese
67a5880c886a1e223b1d57f0
Yingbo Zhou
admin_assigned
2025-02-07T10:45:09.887Z
649bc93758d8b19de0c7785f
/avatars/3ed9473aee23d99f4ee949d3705089ea.svg
Yingbo Zhou
user
yingbozhou
67a5880c886a1e223b1d57f1
Caiming Xiong
admin_assigned
2025-02-07T10:45:02.543Z
649dbcc4e0fff1ed099dc80a
/avatars/c87c273ca628dbcddccbf1ee19b2ce33.svg
Caiming Xiong
user
cxiong
BOLT: Bootstrap Long Chain-of-Thought in Language Models without
  Distillation
Large language models (LLMs), such as o1 from OpenAI, have demonstrated
remarkable reasoning capabilities. o1 generates a long chain-of-thought
(LongCoT) before answering a question. LongCoT allows LLMs to analyze problems,
devise plans, reflect, and backtrack effectively. These actions empower LLM to
solve complex problems. After the release of o1, many teams have attempted to
replicate its LongCoT and reasoning capabilities. In terms of methods, they
primarily rely on knowledge distillation with data from existing models with
LongCoT capacities (e.g., OpenAI-o1, Qwen-QwQ, DeepSeek-R1-Preview), leaving
significant uncertainties on systematically developing such reasoning
abilities. In terms of data domains, these works focus narrowly on math while a
few others include coding, limiting their generalizability. This paper
introduces a novel approach to enable LLM's LongCoT capacity without
distillation from o1-like models or expensive human annotations, where we
bootstrap LongCoT (BOLT) from a standard instruct model. BOLT involves three
stages: 1) LongCoT data bootstrapping with in-context learning on a standard
instruct model; 2) LongCoT supervised finetuning; 3) online training to further
refine LongCoT capacities. In BOLT, only a few in-context examples need to be
constructed during the bootstrapping stage; in our experiments, we created 10
examples, demonstrating the feasibility of this approach. We use
Llama-3.1-70B-Instruct to bootstrap LongCoT and apply our method to various
model scales (7B, 8B, 70B). We achieve impressive performance on a variety of
benchmarks, Arena-Hard, MT-Bench, WildBench, ZebraLogic, MATH500, which
evaluate diverse task-solving and reasoning capabilities.
A method is introduced to enable long chain-of-thought reasoning in large language models using in-context learning and supervised fine-tuning without distillation from pre-existing models or extensive human annotations.
large language models long chain-of-thought LongCoT in-context learning supervised fine-tuning reasoning capabilities bootstrapping Arena-Hard MT-Bench WildBench ZebraLogic MATH500