Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic
  Confidence
Test-Time Scaling (TTS) methods for enhancing Large Language Model (LLM)
reasoning often incur substantial computational costs, primarily due to
extensive reliance on external Process Reward Models (PRMs) or sampling methods
like Best-of-N (BoN). This paper introduces Guided by Gut (GG), an efficient
self-guided TTS framework that achieves PRM-level performance without costly
external verifier models. Our method employs a lightweight tree search guided
solely by intrinsic LLM signals, token-level confidence and step novelty. One
critical innovation is improving the reliability of internal confidence
estimates via a targeted reinforcement learning fine-tuning phase. Empirical
evaluations on challenging mathematical reasoning benchmarks demonstrate that
GG enables smaller models (e.g., 1.5B parameters) to achieve accuracy matching
or surpassing significantly larger models (e.g., 32B-70B parameters), while
reducing GPU memory usage by up to 10x. Compared to PRM-based methods, GG
achieves comparable accuracy with 8x faster inference speeds and 4-5x lower
memory usage. Additionally, GG reduces KV cache memory usage by approximately
50% compared to the BoN strategy, facilitating more efficient and practical
deployment of TTS techniques.
6836abd6871c9d398c136ee0
Amirhosein Ghasemabadi
extracted_confirmed
2025-05-28T07:04:37.500Z
6830c4be7f72826192827659
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/0hTKkO_zytpy9W8_0s291.png
Amirhosein Ghasemabadi
user
AmirhoseinGH
6836abd6871c9d398c136ee1
Keith G. Mills
extracted_confirmed
2025-05-28T06:48:14.969Z
661c391720b47b0daddfcc5a
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RwvDJRtlEEPch27xId9Cb.png
Keith G. Mills
user
kgmills
6836abd6871c9d398c136ee2
Baochun Li
6836abd6871c9d398c136ee3
Di Niu
Guided by Gut: Efficient Test-Time Scaling with Reinforced Intrinsic
  Confidence
Test-Time Scaling (TTS) methods for enhancing Large Language Model (LLM)
reasoning often incur substantial computational costs, primarily due to
extensive reliance on external Process Reward Models (PRMs) or sampling methods
like Best-of-N (BoN). This paper introduces Guided by Gut (GG), an efficient
self-guided TTS framework that achieves PRM-level performance without costly
external verifier models. Our method employs a lightweight tree search guided
solely by intrinsic LLM signals, token-level confidence and step novelty. One
critical innovation is improving the reliability of internal confidence
estimates via a targeted reinforcement learning fine-tuning phase. Empirical
evaluations on challenging mathematical reasoning benchmarks demonstrate that
GG enables smaller models (e.g., 1.5B parameters) to achieve accuracy matching
or surpassing significantly larger models (e.g., 32B-70B parameters), while
reducing GPU memory usage by up to 10x. Compared to PRM-based methods, GG
achieves comparable accuracy with 8x faster inference speeds and 4-5x lower
memory usage. Additionally, GG reduces KV cache memory usage by approximately
50% compared to the BoN strategy, facilitating more efficient and practical
deployment of TTS techniques.
The Guided by Gut (GG) framework enhances LLM reasoning efficiently using intrinsic signals and token-level confidence, outperforming PRM-based methods with faster inference and lower memory usage.
Test-Time Scaling (TTS) Large Language Model (LLM) Process Reward Models (PRMs) Best-of-N (BoN) tree search intrinsic LLM signals token-level confidence step novelty targeted reinforcement learning fine-tuning