A Theoretical Study on Bridging Internal Probability and
  Self-Consistency for LLM Reasoning
Test-time scaling seeks to improve the reasoning performance of large
language models (LLMs) by adding computational resources. A prevalent approach
within the field is sampling-based test-time scaling methods, which enhance
reasoning by generating multiple reasoning paths for a given input during
inference. However, despite its practical success, the theoretical foundations
remain underexplored. In this paper, we provide the first theoretical framework
for analyzing sampling-based test-time scaling methods, grounded in the
perspective of confidence estimation. Based on the framework, we analyze two
dominant paradigms: self-consistency and perplexity, and reveal key
limitations: self-consistency suffers from high estimation error while
perplexity exhibits substantial modeling error and possible degradation of the
estimation error convergence. To address these limitations, we introduce RPC, a
hybrid method that leverages our theoretical insights through two key
components: Perplexity Consistency and Reasoning Pruning. Perplexity
Consistency combines the strengths of self-consistency and perplexity, boosting
the convergence rate of estimation error from linear to exponential while
preserving model error. Reasoning Pruning prevents degradation by eliminating
low-probability reasoning paths. Both theoretical analysis and empirical
results across seven benchmark datasets demonstrate that RPC has a strong
potential for reducing reasoning error. Notably, RPC achieves reasoning
performance comparable to self-consistency while not only enhancing confidence
reliability but also reducing sampling costs by 50%. The code and resources are
available at https://wnjxyk.github.io/RPC.
68f60ee18589920bf4d32293
Zhi Zhou
claimed_verified
2025-10-20T12:09:29.609Z
64675fd0b990713c50317559
https://cdn-avatars.huggingface.co/v1/production/uploads/64675fd0b990713c50317559/qGUiaMyAd4mUjXJe0EXVU.png
Zhi Zhou
user
WNJXYK
68f60ee18589920bf4d32294
Yuhao Tan
68f60ee18589920bf4d32295
Zenan Li
68f60ee18589920bf4d32296
Yuan Yao
68f60ee18589920bf4d32297
Lan-Zhe Guo
68f60ee18589920bf4d32298
Yu-Feng Li
68f60ee18589920bf4d32299
Xiaoxing Ma
A Theoretical Study on Bridging Internal Probability and
  Self-Consistency for LLM Reasoning
Test-time scaling seeks to improve the reasoning performance of large
language models (LLMs) by adding computational resources. A prevalent approach
within the field is sampling-based test-time scaling methods, which enhance
reasoning by generating multiple reasoning paths for a given input during
inference. However, despite its practical success, the theoretical foundations
remain underexplored. In this paper, we provide the first theoretical framework
for analyzing sampling-based test-time scaling methods, grounded in the
perspective of confidence estimation. Based on the framework, we analyze two
dominant paradigms: self-consistency and perplexity, and reveal key
limitations: self-consistency suffers from high estimation error while
perplexity exhibits substantial modeling error and possible degradation of the
estimation error convergence. To address these limitations, we introduce RPC, a
hybrid method that leverages our theoretical insights through two key
components: Perplexity Consistency and Reasoning Pruning. Perplexity
Consistency combines the strengths of self-consistency and perplexity, boosting
the convergence rate of estimation error from linear to exponential while
preserving model error. Reasoning Pruning prevents degradation by eliminating
low-probability reasoning paths. Both theoretical analysis and empirical
results across seven benchmark datasets demonstrate that RPC has a strong
potential for reducing reasoning error. Notably, RPC achieves reasoning
performance comparable to self-consistency while not only enhancing confidence
reliability but also reducing sampling costs by 50%. The code and resources are
available at https://wnjxyk.github.io/RPC.
A theoretical framework for sampling-based test-time scaling in large language models reveals limitations in self-consistency and perplexity, and introduces RPC to improve reasoning performance and reduce sampling costs.
sampling-based test-time scaling large language models self-consistency perplexity Perplexity Consistency Reasoning Pruning confidence estimation reasoning error benchmark datasets