Towards System 2 Reasoning in LLMs: Learning How to Think With Meta
  Chain-of-Though
We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends
traditional Chain-of-Thought (CoT) by explicitly modeling the underlying
reasoning required to arrive at a particular CoT. We present empirical evidence
from state-of-the-art models exhibiting behaviors consistent with in-context
search, and explore methods for producing Meta-CoT via process supervision,
synthetic data generation, and search algorithms. Finally, we outline a
concrete pipeline for training a model to produce Meta-CoTs, incorporating
instruction tuning with linearized search traces and reinforcement learning
post-training. Finally, we discuss open research questions, including scaling
laws, verifier roles, and the potential for discovering novel reasoning
algorithms. This work provides a theoretical and practical roadmap to enable
Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in
artificial intelligence.
677f4e0d05ab3422540b88ff
Violet Xiang
claimed_verified
2025-01-15T08:49:18.274Z
63a4c25f769ff94bc94ec301
/avatars/60fd588b972eecca9d74636a244047a1.svg
Violet Xiang
user
violetxi
677f4e0d05ab3422540b8900
Charlie Snell
677f4e0d05ab3422540b8901
Kanishk Gandhi
claimed_verified
2025-01-10T08:21:32.915Z
63e6a880f2e9a8f22c5a1630
/avatars/53b57690fe052ce6882bbfc87b11567c.svg
Kanishk Gandhi
user
obiwan96
677f4e0d05ab3422540b8902
Alon Albalak
claimed_verified
2025-01-09T19:57:57.972Z
611a7ec4289467cafea62d13
https://cdn-avatars.huggingface.co/v1/production/uploads/611a7ec4289467cafea62d13/pck-0fmPQkoU7yzh6-WoL.jpeg
Alon Albalak
user
alon-albalak
677f4e0d05ab3422540b8903
Anikait Singh
claimed_verified
2025-01-09T10:06:59.745Z
6511ee845b7e52b0251fdee9
https://cdn-avatars.huggingface.co/v1/production/uploads/6511ee845b7e52b0251fdee9/hTIwiIYBGOVnIrxtpri83.png
Anikait Singh
user
Asap7772
677f4e0d05ab3422540b8904
Chase Blagden
677f4e0d05ab3422540b8905
Duy Phung
677f4e0d05ab3422540b8906
Rafael Rafailov
677f4e0d05ab3422540b8907
Nathan Lile
claimed_verified
2025-01-09T10:07:02.418Z
61aa15fd8a9625ebfe284286
https://cdn-avatars.huggingface.co/v1/production/uploads/61aa15fd8a9625ebfe284286/KaGzIeijcgcN15JErCqft.jpeg
nathan lile
user
nlile
677f4e0d05ab3422540b8908
Dakota Mahan
677f4e0d05ab3422540b8909
Louis Castricato
claimed_verified
2025-01-13T08:54:36.509Z
60d35a0307da9c17c7270909
https://cdn-avatars.huggingface.co/v1/production/uploads/1665155571839-60d35a0307da9c17c7270909.jpeg
Louis Castricato
user
LouisCastricato
677f4e0d05ab3422540b890a
Jan-Philipp Franken
677f4e0d05ab3422540b890b
Nick Haber
677f4e0d05ab3422540b890c
Chelsea Finn
Towards System 2 Reasoning in LLMs: Learning How to Think With Meta
  Chain-of-Though
We propose a novel framework, Meta Chain-of-Thought (Meta-CoT), which extends
traditional Chain-of-Thought (CoT) by explicitly modeling the underlying
reasoning required to arrive at a particular CoT. We present empirical evidence
from state-of-the-art models exhibiting behaviors consistent with in-context
search, and explore methods for producing Meta-CoT via process supervision,
synthetic data generation, and search algorithms. Finally, we outline a
concrete pipeline for training a model to produce Meta-CoTs, incorporating
instruction tuning with linearized search traces and reinforcement learning
post-training. Finally, we discuss open research questions, including scaling
laws, verifier roles, and the potential for discovering novel reasoning
algorithms. This work provides a theoretical and practical roadmap to enable
Meta-CoT in LLMs, paving the way for more powerful and human-like reasoning in
artificial intelligence.
The Meta Chain-of-Thought framework extends traditional CoT by explicitly modeling reasoning, enabling more powerful and human-like reasoning in LLMs through process supervision, synthetic data generation, and reinforcement learning.
Meta Chain-of-Thought Meta-CoT Chain-of-Thought CoT in-context search process supervision synthetic data generation search algorithms instruction tuning linearized search traces reinforcement learning scaling laws verifier roles reasoning algorithms LLMs