Learning to Reason via Mixture-of-Thought for Logical Reasoning
Human beings naturally utilize multiple reasoning modalities to learn and
solve logical problems, i.e., different representational formats such as
natural language, code, and symbolic logic. In contrast, most existing
LLM-based approaches operate with a single reasoning modality during training,
typically natural language. Although some methods explored modality selection
or augmentation at inference time, the training process remains modality-blind,
limiting synergy among modalities. To fill in this gap, we propose
Mixture-of-Thought (MoT), a framework that enables LLMs to reason across three
complementary modalities: natural language, code, and a newly introduced
symbolic modality, truth-table, which systematically enumerates logical cases
and partially mitigates key failure modes in natural language reasoning. MoT
adopts a two-phase design: (1) self-evolving MoT training, which jointly learns
from filtered, self-generated rationales across modalities; and (2) MoT
inference, which fully leverages the synergy of three modalities to produce
better predictions. Experiments on logical reasoning benchmarks including FOLIO
and ProofWriter demonstrate that our MoT framework consistently and
significantly outperforms strong LLM baselines with single-modality
chain-of-thought approaches, achieving up to +11.7pp average accuracy gain.
Further analyses show that our MoT framework benefits both training and
inference stages; that it is particularly effective on harder logical reasoning
problems; and that different modalities contribute complementary strengths,
with truth-table reasoning helping to overcome key bottlenecks in natural
language inference.
682ec132fbbbc6e3a91b106e
Tong Zheng
claimed_verified
2025-05-22T07:15:27.903Z
6623ea65b642e29cdf90a1b4
/avatars/e32e90574c1162b2be87ed78604e3e4d.svg
TongZheng
user
TongZheng1999
682ec132fbbbc6e3a91b106f
Lichang Chen
claimed_verified
2025-05-27T07:55:20.088Z
62323bb408bcea92917e42ee
https://cdn-avatars.huggingface.co/v1/production/uploads/62323bb408bcea92917e42ee/2vHxkv-oSROtLteOnqa8P.jpeg
Lichang Chen
user
Lichang-Chen
682ec132fbbbc6e3a91b1070
Simeng Han
claimed_verified
2025-05-26T08:19:40.053Z
617ad22b941993035fbaf2b1
/avatars/3a4b9b06b85dc9be824c0c70ecd4a37f.svg
Simeng Han
user
Simeng
682ec132fbbbc6e3a91b1071
R. Thomas McCoy
682ec132fbbbc6e3a91b1072
Heng Huang
Learning to Reason via Mixture-of-Thought for Logical Reasoning
Human beings naturally utilize multiple reasoning modalities to learn and
solve logical problems, i.e., different representational formats such as
natural language, code, and symbolic logic. In contrast, most existing
LLM-based approaches operate with a single reasoning modality during training,
typically natural language. Although some methods explored modality selection
or augmentation at inference time, the training process remains modality-blind,
limiting synergy among modalities. To fill in this gap, we propose
Mixture-of-Thought (MoT), a framework that enables LLMs to reason across three
complementary modalities: natural language, code, and a newly introduced
symbolic modality, truth-table, which systematically enumerates logical cases
and partially mitigates key failure modes in natural language reasoning. MoT
adopts a two-phase design: (1) self-evolving MoT training, which jointly learns
from filtered, self-generated rationales across modalities; and (2) MoT
inference, which fully leverages the synergy of three modalities to produce
better predictions. Experiments on logical reasoning benchmarks including FOLIO
and ProofWriter demonstrate that our MoT framework consistently and
significantly outperforms strong LLM baselines with single-modality
chain-of-thought approaches, achieving up to +11.7pp average accuracy gain.
Further analyses show that our MoT framework benefits both training and
inference stages; that it is particularly effective on harder logical reasoning
problems; and that different modalities contribute complementary strengths,
with truth-table reasoning helping to overcome key bottlenecks in natural
language inference.
A Mixture-of-Thought framework enables LLMs to reason across natural language, code, and symbolic logic, improving accuracy on logical reasoning tasks compared to single-modality approaches.
LMM-based approaches reasoning modality natural language code symbolic logic modality-blind self-evolving MoT training MoT inference FOLIO ProofWriter truth-table logical reasoning