StyleBench: Evaluating thinking styles in Large Language Models
The effectiveness of Large Language Models (LLMs) is heavily influenced by
the reasoning strategies, or styles of thought, employed in their prompts.
However, the interplay between these reasoning styles, model architecture, and
task type remains poorly understood. To address this, we introduce StyleBench,
a comprehensive benchmark for systematically evaluating reasoning styles across
diverse tasks and models. We assess five representative reasoning styles,
including Chain of Thought (CoT), Tree of Thought (ToT), Algorithm of Thought
(AoT), Sketch of Thought (SoT), and Chain-of-Draft (CoD) on five reasoning
tasks, using 15 open-source models from major families (LLaMA, Qwen, Mistral,
Gemma, GPT-OSS, Phi, and DeepSeek) ranging from 270M to 120B parameters. Our
large-scale analysis reveals that no single style is universally optimal. We
demonstrate that strategy efficacy is highly contingent on both model scale and
task type: search-based methods (AoT, ToT) excel in open-ended problems but
require large-scale models, while concise styles (SoT, CoD) achieve radical
efficiency gains on well-defined tasks. Furthermore, we identify key behavioral
patterns: smaller models frequently fail to follow output instructions and
default to guessing, while reasoning robustness emerges as a function of scale.
Our findings offer a crucial roadmap for selecting optimal reasoning strategies
based on specific constraints, we open source the benchmark in
https://github.com/JamesJunyuGuo/Style_Bench.
68d5f5278ccd91bdd39ffda4
Junyu Guo
68d5f5278ccd91bdd39ffda5
Shangding Gu
admin_assigned
2025-09-26T16:14:38.370Z
668c952f142f9b26a4a17e0f
/avatars/adfc300c2ed1155af5248754fb59ee72.svg
Shangding Gu
user
Shangding-B
68d5f5278ccd91bdd39ffda6
Ming Jin
68d5f5278ccd91bdd39ffda7
Costas Spanos
68d5f5278ccd91bdd39ffda8
Javad Lavaei
StyleBench: Evaluating thinking styles in Large Language Models
The effectiveness of Large Language Models (LLMs) is heavily influenced by
the reasoning strategies, or styles of thought, employed in their prompts.
However, the interplay between these reasoning styles, model architecture, and
task type remains poorly understood. To address this, we introduce StyleBench,
a comprehensive benchmark for systematically evaluating reasoning styles across
diverse tasks and models. We assess five representative reasoning styles,
including Chain of Thought (CoT), Tree of Thought (ToT), Algorithm of Thought
(AoT), Sketch of Thought (SoT), and Chain-of-Draft (CoD) on five reasoning
tasks, using 15 open-source models from major families (LLaMA, Qwen, Mistral,
Gemma, GPT-OSS, Phi, and DeepSeek) ranging from 270M to 120B parameters. Our
large-scale analysis reveals that no single style is universally optimal. We
demonstrate that strategy efficacy is highly contingent on both model scale and
task type: search-based methods (AoT, ToT) excel in open-ended problems but
require large-scale models, while concise styles (SoT, CoD) achieve radical
efficiency gains on well-defined tasks. Furthermore, we identify key behavioral
patterns: smaller models frequently fail to follow output instructions and
default to guessing, while reasoning robustness emerges as a function of scale.
Our findings offer a crucial roadmap for selecting optimal reasoning strategies
based on specific constraints, we open source the benchmark in
https://github.com/JamesJunyuGuo/Style_Bench.
StyleBench evaluates various reasoning styles across tasks and models, revealing that strategy efficacy depends on model scale and task type.
Large Language Models (LLMs) Chain of Thought (CoT) Tree of Thought (ToT) Algorithm of Thought (AoT) Sketch of Thought (SoT) Chain-of-Draft (CoD) StyleBench open-source models LLaMA Qwen Mistral Gemma GPT-OSS Phi DeepSeek reasoning robustness