System-1.5 Reasoning: Traversal in Language and Latent Spaces with
  Dynamic Shortcuts
Chain-of-thought (CoT) reasoning enables large language models (LLMs) to move
beyond fast System-1 responses and engage in deliberative System-2 reasoning.
However, this comes at the cost of significant inefficiency due to verbose
intermediate output. Recent latent-space reasoning methods improve efficiency
by operating on hidden states without decoding into language, yet they treat
all steps uniformly, failing to distinguish critical deductions from auxiliary
steps and resulting in suboptimal use of computational resources. In this
paper, we propose System-1.5 Reasoning, an adaptive reasoning framework that
dynamically allocates computation across reasoning steps through shortcut paths
in latent space. Specifically, System-1.5 Reasoning introduces two types of
dynamic shortcuts. The model depth shortcut (DS) adaptively reasons along the
vertical depth by early exiting non-critical tokens through lightweight adapter
branches, while allowing critical tokens to continue through deeper Transformer
layers. The step shortcut (SS) reuses hidden states across the decoding steps
to skip trivial steps and reason horizontally in latent space. Training
System-1.5 Reasoning involves a two-stage self-distillation process: first
distilling natural language CoT into latent-space continuous thought, and then
distilling full-path System-2 latent reasoning into adaptive shortcut paths
(System-1.5 Reasoning). Experiments on reasoning tasks demonstrate the superior
performance of our method. For example, on GSM8K, System-1.5 Reasoning achieves
reasoning performance comparable to traditional CoT fine-tuning methods while
accelerating inference by over 20x and reducing token generation by 92.31% on
average.
6839ca4456bcc85d9fc05477
Xiaoqiang Wang
6839ca4456bcc85d9fc05478
Suyuchen Wang
claimed_verified
2025-06-02T07:45:07.858Z
62bb1e0f3ff437e49a3088e5
https://cdn-avatars.huggingface.co/v1/production/uploads/62bb1e0f3ff437e49a3088e5/MWNanci3x5g780xh-704U.png
Suyuchen Wang
user
sheryc
6839ca4456bcc85d9fc05479
Yun Zhu
6839ca4456bcc85d9fc0547a
Bang Liu
System-1.5 Reasoning: Traversal in Language and Latent Spaces with
  Dynamic Shortcuts
Chain-of-thought (CoT) reasoning enables large language models (LLMs) to move
beyond fast System-1 responses and engage in deliberative System-2 reasoning.
However, this comes at the cost of significant inefficiency due to verbose
intermediate output. Recent latent-space reasoning methods improve efficiency
by operating on hidden states without decoding into language, yet they treat
all steps uniformly, failing to distinguish critical deductions from auxiliary
steps and resulting in suboptimal use of computational resources. In this
paper, we propose System-1.5 Reasoning, an adaptive reasoning framework that
dynamically allocates computation across reasoning steps through shortcut paths
in latent space. Specifically, System-1.5 Reasoning introduces two types of
dynamic shortcuts. The model depth shortcut (DS) adaptively reasons along the
vertical depth by early exiting non-critical tokens through lightweight adapter
branches, while allowing critical tokens to continue through deeper Transformer
layers. The step shortcut (SS) reuses hidden states across the decoding steps
to skip trivial steps and reason horizontally in latent space. Training
System-1.5 Reasoning involves a two-stage self-distillation process: first
distilling natural language CoT into latent-space continuous thought, and then
distilling full-path System-2 latent reasoning into adaptive shortcut paths
(System-1.5 Reasoning). Experiments on reasoning tasks demonstrate the superior
performance of our method. For example, on GSM8K, System-1.5 Reasoning achieves
reasoning performance comparable to traditional CoT fine-tuning methods while
accelerating inference by over 20x and reducing token generation by 92.31% on
average.
System-1.5 Reasoning improves the efficiency and performance of large language models by dynamically allocating computation through adaptive shortcuts in latent space, leading to faster inference and reduced token generation.
Chain-of-thought (CoT) reasoning System-1 reasoning System-2 reasoning latent-space reasoning hidden states adaptive reasoning framework model depth shortcut (DS) step shortcut (SS) lightweight adapter branches Transformer layers self-distillation process continuous thought adaptive shortcut paths GSM8K CoT fine-tuning methods