Self-Discover: Large Language Models Self-Compose Reasoning Structures
We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the
task-intrinsic reasoning structures to tackle complex reasoning problems that
are challenging for typical prompting methods. Core to the framework is a
self-discovery process where LLMs select multiple atomic reasoning modules such
as critical thinking and step-by-step thinking, and compose them into an
explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER
substantially improves GPT-4 and PaLM 2's performance on challenging reasoning
benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as
much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER
outperforms inference-intensive methods such as CoT-Self-Consistency by more
than 20%, while requiring 10-40x fewer inference compute. Finally, we show that
the self-discovered reasoning structures are universally applicable across
model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share
commonalities with human reasoning patterns.
65c2ef81c1ab7dcfc9594950
Pei Zhou
extracted_pending
2024-02-07T02:48:33.652Z
60844fd056f7623a0b84914b
/avatars/f7d2763258b8c614783ead9480fbcf8d.svg
Pei Zhou
user
peiz
65c2ef81c1ab7dcfc9594951
Jay Pujara
65c2ef81c1ab7dcfc9594952
Xiang Ren
65c2ef81c1ab7dcfc9594953
Xinyun Chen
65c2ef81c1ab7dcfc9594954
Heng-Tze Cheng
65c2ef81c1ab7dcfc9594955
Quoc V. Le
65c2ef81c1ab7dcfc9594956
Ed H. Chi
65c2ef81c1ab7dcfc9594957
Denny Zhou
claimed_verified
2024-02-20T18:27:07.442Z
65d4d62d493aeaf0bb295f1b
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/vXMxaoWwFziul8TqIhQw8.jpeg
Denny Zhou
user
dennyzhou
65c2ef81c1ab7dcfc9594958
Swaroop Mishra
65c2ef81c1ab7dcfc9594959
Huaixiu Steven Zheng
Self-Discover: Large Language Models Self-Compose Reasoning Structures
We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the
task-intrinsic reasoning structures to tackle complex reasoning problems that
are challenging for typical prompting methods. Core to the framework is a
self-discovery process where LLMs select multiple atomic reasoning modules such
as critical thinking and step-by-step thinking, and compose them into an
explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER
substantially improves GPT-4 and PaLM 2's performance on challenging reasoning
benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as
much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER
outperforms inference-intensive methods such as CoT-Self-Consistency by more
than 20%, while requiring 10-40x fewer inference compute. Finally, we show that
the self-discovered reasoning structures are universally applicable across
model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share
commonalities with human reasoning patterns.
SELF-DISCOVER is a framework enabling LLMs to generate task-specific reasoning structures, enhancing performance on complex reasoning tasks with reduced computational cost.
SELF-DISCOVER LLMs self-discovery process atomic reasoning modules critical thinking step-by-step thinking BigBench-Hard grounded agent reasoning MATH Chain of Thought (CoT) CoT-Self-Consistency Llama2