Efficient Test-Time Scaling via Self-Calibration
Increasing test-time computation is a straightforward approach to enhancing
the quality of responses in Large Language Models (LLMs). While Best-of-N
sampling and Self-Consistency with majority voting are simple and effective,
they require a fixed number of sampling responses for each query, regardless of
its complexity. This could result in wasted computation for simpler questions
and insufficient exploration for more challenging ones. In this work, we argue
that model confidence of responses can be used for improving the efficiency of
test-time scaling. Unfortunately, LLMs are known to be overconfident and
provide unreliable confidence estimation. To address this limitation, we
introduce Self-Calibration by distilling Self-Consistency-derived confidence
into the model itself. This enables reliable confidence estimation at test time
with one forward pass. We then design confidence-based efficient test-time
scaling methods to handle queries of various difficulty, such as Early-Stopping
for Best-of-N and Self-Consistency with calibrated confidence. Experiments on
three LLMs across six datasets demonstrate the effectiveness of our approach.
Specifically, applying confidence-based Early Stopping to Best-of-N improves
MathQA accuracy from 81.0 to 83.6 with a sample budget of 16 responses,
indicating the efficacy of confidence-based sampling strategy at inference
time.
67c732c14aaf26f75cea0d82
Chengsong Huang
claimed_verified
2025-03-04T21:15:36.013Z
62ea79dd01ed9b0e8f61ccd3
/avatars/70af83e0e267be39fcd5f23b85e2dafa.svg
Chengsong Huang
user
ChengsongHuang
67c732c14aaf26f75cea0d83
Langlin Huang
67c732c14aaf26f75cea0d84
Jixuan Leng
67c732c14aaf26f75cea0d85
Jiacheng Liu
claimed_verified
2025-04-10T06:47:21.573Z
635f46d1928a42bc95cfcf7c
https://cdn-avatars.huggingface.co/v1/production/uploads/635f46d1928a42bc95cfcf7c/5KF8aLiDCJdl7B1SdJ-7V.png
Jiacheng Liu
user
liujch1998
67c732c14aaf26f75cea0d86
Jiaxin Huang
Efficient Test-Time Scaling via Self-Calibration
Increasing test-time computation is a straightforward approach to enhancing
the quality of responses in Large Language Models (LLMs). While Best-of-N
sampling and Self-Consistency with majority voting are simple and effective,
they require a fixed number of sampling responses for each query, regardless of
its complexity. This could result in wasted computation for simpler questions
and insufficient exploration for more challenging ones. In this work, we argue
that model confidence of responses can be used for improving the efficiency of
test-time scaling. Unfortunately, LLMs are known to be overconfident and
provide unreliable confidence estimation. To address this limitation, we
introduce Self-Calibration by distilling Self-Consistency-derived confidence
into the model itself. This enables reliable confidence estimation at test time
with one forward pass. We then design confidence-based efficient test-time
scaling methods to handle queries of various difficulty, such as Early-Stopping
for Best-of-N and Self-Consistency with calibrated confidence. Experiments on
three LLMs across six datasets demonstrate the effectiveness of our approach.
Specifically, applying confidence-based Early Stopping to Best-of-N improves
MathQA accuracy from 81.0 to 83.6 with a sample budget of 16 responses,
indicating the efficacy of confidence-based sampling strategy at inference
time.
Self-Calibration enhances the efficiency of test-time scaling in Large Language Models by using model confidence to dynamically adjust sampling strategies.
Large Language Models (LLMs) Best-of-N sampling Self-Consistency Self-Calibration confidence estimation Early-Stopping MathQA