SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior
  Reasoning LLMs
Recent work shows that, beyond discrete reasoning through explicit
chain-of-thought steps, which are limited by the boundaries of natural
languages, large language models (LLMs) can also reason continuously in latent
space, allowing richer information per step and thereby improving token
efficiency. Despite this promise, latent reasoning still faces two challenges,
especially in training-free settings: 1) purely latent reasoning broadens the
search distribution by maintaining multiple implicit paths, which diffuses
probability mass, introduces noise, and impedes convergence to a single
high-confidence solution, thereby hurting accuracy; and 2) overthinking
persists even without explicit text, wasting tokens and degrading efficiency.
To address these issues, we introduce SwiReasoning, a training-free framework
for LLM reasoning which features two key innovations: 1) SwiReasoning
dynamically switches between explicit and latent reasoning, guided by
block-wise confidence estimated from entropy trends in next-token
distributions, to balance exploration and exploitation and promote timely
convergence. 2) By limiting the maximum number of thinking-block switches,
SwiReasoning curbs overthinking and improves token efficiency across varying
problem difficulties. On widely used mathematics and STEM benchmarks,
SwiReasoning consistently improves average accuracy by 1.5%-2.8% across
reasoning LLMs of different model families and scales. Furthermore, under
constrained budgets, SwiReasoning improves average token efficiency by 56%-79%,
with larger gains as budgets tighten.
68e48451e4e093a7044e4d6c
Dachuan Shi
claimed_verified
2025-10-07T12:26:41.431Z
6482c4178ad9934217a4287d
/avatars/b852d0ced7d98720606b99adaac596f0.svg
Dachuan Shi
user
sdc17
68e48451e4e093a7044e4d6d
Abedelkadir Asi
68e48451e4e093a7044e4d6e
Keying Li
68e48451e4e093a7044e4d6f
Xiangchi Yuan
68e48451e4e093a7044e4d70
Leyan Pan
68e48451e4e093a7044e4d71
Wenke Lee
68e48451e4e093a7044e4d72
Wen Xiao
SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior
  Reasoning LLMs
Recent work shows that, beyond discrete reasoning through explicit
chain-of-thought steps, which are limited by the boundaries of natural
languages, large language models (LLMs) can also reason continuously in latent
space, allowing richer information per step and thereby improving token
efficiency. Despite this promise, latent reasoning still faces two challenges,
especially in training-free settings: 1) purely latent reasoning broadens the
search distribution by maintaining multiple implicit paths, which diffuses
probability mass, introduces noise, and impedes convergence to a single
high-confidence solution, thereby hurting accuracy; and 2) overthinking
persists even without explicit text, wasting tokens and degrading efficiency.
To address these issues, we introduce SwiReasoning, a training-free framework
for LLM reasoning which features two key innovations: 1) SwiReasoning
dynamically switches between explicit and latent reasoning, guided by
block-wise confidence estimated from entropy trends in next-token
distributions, to balance exploration and exploitation and promote timely
convergence. 2) By limiting the maximum number of thinking-block switches,
SwiReasoning curbs overthinking and improves token efficiency across varying
problem difficulties. On widely used mathematics and STEM benchmarks,
SwiReasoning consistently improves average accuracy by 1.5%-2.8% across
reasoning LLMs of different model families and scales. Furthermore, under
constrained budgets, SwiReasoning improves average token efficiency by 56%-79%,
with larger gains as budgets tighten.
SwiReasoning, a training-free framework for LLMs, dynamically switches between explicit and latent reasoning to improve accuracy and token efficiency.
latent reasoning explicit reasoning next-token distributions entropy trends block-wise confidence thinking-block switches token efficiency mathematics benchmarks STEM benchmarks