ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
We present that hierarchical LLM reasoning via scaling thought templates can
effectively optimize the reasoning search space and outperform the mathematical
reasoning capabilities of powerful LLMs like OpenAI o1-preview and DeepSeek V3.
We train our ReasonFlux-32B model with only 8 GPUs and introduces three
innovations: (i) a structured and generic thought template library, containing
around 500 high-level thought templates capable of generalizing to similar or
relevant reasoning problems; (ii) performing hierarchical reinforcement
learning on a sequence of thought templates instead of long CoTs, optimizing a
base LLM to plan out an optimal template trajectory for gradually handling
complex problems; (iii) a brand new inference scaling system that enables
hierarchical LLM reasoning by adaptively scaling thought templates at inference
time. With a template trajectory containing sequential thought templates, our
ReasonFlux-32B significantly advances math reasoning capabilities to
state-of-the-art levels. Notably, on the MATH benchmark, it achieves an
accuracy of 91.2% and surpasses o1-preview by 6.7%. On the USA Math Olympiad
(AIME) benchmark, ReasonFlux-32B solves an average of 56.7% of problems,
surpassing o1-preview and DeepSeek-V3 by 27% and 45%, respectively. Code:
https://github.com/Gen-Verse/ReasonFlux
67aac8adfe33f6d8d695bc40
Ling Yang
claimed_verified
2025-02-11T14:25:31.970Z
64fde4e252e82dd432b74ce9
https://cdn-avatars.huggingface.co/v1/production/uploads/64fde4e252e82dd432b74ce9/-CQZbBP7FsPPyawYrsi4z.jpeg
Ling Yang
user
Lingaaaaaaa
67aac8adfe33f6d8d695bc41
Zhaochen Yu
67aac8adfe33f6d8d695bc42
Bin Cui
67aac8adfe33f6d8d695bc43
Mengdi Wang
ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates
We present that hierarchical LLM reasoning via scaling thought templates can
effectively optimize the reasoning search space and outperform the mathematical
reasoning capabilities of powerful LLMs like OpenAI o1-preview and DeepSeek V3.
We train our ReasonFlux-32B model with only 8 GPUs and introduces three
innovations: (i) a structured and generic thought template library, containing
around 500 high-level thought templates capable of generalizing to similar or
relevant reasoning problems; (ii) performing hierarchical reinforcement
learning on a sequence of thought templates instead of long CoTs, optimizing a
base LLM to plan out an optimal template trajectory for gradually handling
complex problems; (iii) a brand new inference scaling system that enables
hierarchical LLM reasoning by adaptively scaling thought templates at inference
time. With a template trajectory containing sequential thought templates, our
ReasonFlux-32B significantly advances math reasoning capabilities to
state-of-the-art levels. Notably, on the MATH benchmark, it achieves an
accuracy of 91.2% and surpasses o1-preview by 6.7%. On the USA Math Olympiad
(AIME) benchmark, ReasonFlux-32B solves an average of 56.7% of problems,
surpassing o1-preview and DeepSeek-V3 by 27% and 45%, respectively. Code:
https://github.com/Gen-Verse/ReasonFlux
Hierarchical reasoning with LLMs using scaled thought templates improves mathematical reasoning and outperforms existing models on benchmarks like MATH and AIME.
hierarchical LLM reasoning thought templates mathematical reasoning reinforcement learning template trajectory inference scaling system ReasonFlux-32B MATH benchmark USA Math Olympiad AIME benchmark