Deep Self-Evolving Reasoning
Long-form chain-of-thought reasoning has become a cornerstone of advanced
reasoning in large language models. While recent verification-refinement
frameworks have enabled proprietary models to solve Olympiad-level problems,
their effectiveness hinges on strong, reliable verification and correction
capabilities, which remain fragile in open-weight, smaller-scale models. This
work demonstrates that even with weak verification and refinement capabilities
on hard tasks, the reasoning limits of such models can be substantially
extended through a probabilistic paradigm we call Deep Self-Evolving Reasoning
(DSER). We conceptualize iterative reasoning as a Markov chain, where each step
represents a stochastic transition in the solution space. The key insight is
that convergence to a correct solution is guaranteed as long as the probability
of improvement marginally exceeds that of degradation. By running multiple
long-horizon, self-evolving processes in parallel, DSER amplifies these small
positive tendencies, enabling the model to asymptotically approach correct
answers. Empirically, we apply DSER to the DeepSeek-R1-0528-Qwen3-8B model. On
the challenging AIME 2024-2025 benchmark, DSER solves 5 out of 9 previously
unsolvable problems and boosts overall performance, enabling this compact model
to surpass the single-turn accuracy of its 600B-parameter teacher through
majority voting. Beyond its immediate utility for test-time scaling, the DSER
framework serves to diagnose the fundamental limitations of current open-weight
reasoners. By clearly delineating their shortcomings in self-verification,
refinement, and stability, our findings establish a clear research agenda for
developing next-generation models with powerful, intrinsic self-evolving
capabilities.
68f6f00024c448936311187a
Zihan Liu
68f6f00024c448936311187b
Shun Zheng
claimed_verified
2025-10-21T14:03:47.838Z
64a7a2bad001860e0c34f7f2
/avatars/927c71448c3f8a65532b387196a82f61.svg
Shun Zheng
user
shun-zheng
68f6f00024c448936311187c
Xumeng Wen
68f6f00024c448936311187d
Yang Wang
68f6f00024c448936311187e
Jiang Bian
68f6f00024c448936311187f
Mao Yang
Deep Self-Evolving Reasoning
Long-form chain-of-thought reasoning has become a cornerstone of advanced
reasoning in large language models. While recent verification-refinement
frameworks have enabled proprietary models to solve Olympiad-level problems,
their effectiveness hinges on strong, reliable verification and correction
capabilities, which remain fragile in open-weight, smaller-scale models. This
work demonstrates that even with weak verification and refinement capabilities
on hard tasks, the reasoning limits of such models can be substantially
extended through a probabilistic paradigm we call Deep Self-Evolving Reasoning
(DSER). We conceptualize iterative reasoning as a Markov chain, where each step
represents a stochastic transition in the solution space. The key insight is
that convergence to a correct solution is guaranteed as long as the probability
of improvement marginally exceeds that of degradation. By running multiple
long-horizon, self-evolving processes in parallel, DSER amplifies these small
positive tendencies, enabling the model to asymptotically approach correct
answers. Empirically, we apply DSER to the DeepSeek-R1-0528-Qwen3-8B model. On
the challenging AIME 2024-2025 benchmark, DSER solves 5 out of 9 previously
unsolvable problems and boosts overall performance, enabling this compact model
to surpass the single-turn accuracy of its 600B-parameter teacher through
majority voting. Beyond its immediate utility for test-time scaling, the DSER
framework serves to diagnose the fundamental limitations of current open-weight
reasoners. By clearly delineating their shortcomings in self-verification,
refinement, and stability, our findings establish a clear research agenda for
developing next-generation models with powerful, intrinsic self-evolving
capabilities.
Deep Self-Evolving Reasoning (DSER) extends the reasoning capabilities of smaller models by iteratively improving solutions through a probabilistic Markov chain, enabling them to solve previously unsolvable problems and surpass larger models in accuracy.
Deep Self-Evolving Reasoning DSER Markov chain iterative reasoning probabilistic paradigm self-evolving processes AIME 2024-2025 benchmark majority voting self-verification refinement stability