Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time
  Scaling
Test-Time Scaling (TTS) is an important method for improving the performance
of Large Language Models (LLMs) by using additional computation during the
inference phase. However, current studies do not systematically analyze how
policy models, Process Reward Models (PRMs), and problem difficulty influence
TTS. This lack of analysis limits the understanding and practical use of TTS
methods. In this paper, we focus on two core questions: (1) What is the optimal
approach to scale test-time computation across different policy models, PRMs,
and problem difficulty levels? (2) To what extent can extended computation
improve the performance of LLMs on complex tasks, and can smaller language
models outperform larger ones through this approach? Through comprehensive
experiments on MATH-500 and challenging AIME24 tasks, we have the following
observations: (1) The compute-optimal TTS strategy is highly dependent on the
choice of policy model, PRM, and problem difficulty. (2) With our
compute-optimal TTS strategy, extremely small policy models can outperform
larger models. For example, a 1B LLM can exceed a 405B LLM on MATH-500.
Moreover, on both MATH-500 and AIME24, a 0.5B LLM outperforms GPT-4o, a 3B LLM
surpasses a 405B LLM, and a 7B LLM beats o1 and DeepSeek-R1, while with higher
inference efficiency. These findings show the significance of adapting TTS
strategies to the specific characteristics of each task and model and indicate
that TTS is a promising approach for enhancing the reasoning abilities of LLMs.
67aabf93c0f8648f68c68ce4
Runze Liu
claimed_verified
2025-02-11T07:55:22.940Z
667187ba9ab144eb3ac43a1b
/avatars/db5558aa1c5160b9aee8b58573271959.svg
Runze Liu
user
RyanLiu112
67aabf93c0f8648f68c68ce5
Junqi Gao
admin_assigned
2025-02-11T15:54:46.128Z
67ab05fe4c6ca2d5db4c0c52
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/QpGUNDkeuKjX71s2GXlXF.png
Junqi Gao
user
ChetKao
67aabf93c0f8648f68c68ce6
Jian Zhao
67aabf93c0f8648f68c68ce7
Kaiyan Zhang
claimed_verified
2025-02-11T07:55:18.725Z
60bc94cd85a3ab33829b6211
/avatars/b57d36c7577fbbb42ea5b963eef4144a.svg
Kaiyan Zhang
user
iseesaw
67aabf93c0f8648f68c68ce8
Xiu Li
67aabf93c0f8648f68c68ce9
Biqing Qi
admin_assigned
2025-02-11T15:55:23.328Z
645d9c3058f9ee315148116d
https://cdn-avatars.huggingface.co/v1/production/uploads/645d9c3058f9ee315148116d/uBoAWgrF2Di4WcXVGW9fP.jpeg
Biqing Qi
user
jackqi7
67aabf93c0f8648f68c68cea
Wanli Ouyang
67aabf93c0f8648f68c68ceb
Bowen Zhou
admin_assigned
2025-02-11T15:55:11.315Z
669f614b59adf5b56e05bce3
/avatars/ffd4189efbceb0e63a03db273065a44b.svg
BowenZhou
user
bowenZhou
Can 1B LLM Surpass 405B LLM? Rethinking Compute-Optimal Test-Time
  Scaling
Test-Time Scaling (TTS) is an important method for improving the performance
of Large Language Models (LLMs) by using additional computation during the
inference phase. However, current studies do not systematically analyze how
policy models, Process Reward Models (PRMs), and problem difficulty influence
TTS. This lack of analysis limits the understanding and practical use of TTS
methods. In this paper, we focus on two core questions: (1) What is the optimal
approach to scale test-time computation across different policy models, PRMs,
and problem difficulty levels? (2) To what extent can extended computation
improve the performance of LLMs on complex tasks, and can smaller language
models outperform larger ones through this approach? Through comprehensive
experiments on MATH-500 and challenging AIME24 tasks, we have the following
observations: (1) The compute-optimal TTS strategy is highly dependent on the
choice of policy model, PRM, and problem difficulty. (2) With our
compute-optimal TTS strategy, extremely small policy models can outperform
larger models. For example, a 1B LLM can exceed a 405B LLM on MATH-500.
Moreover, on both MATH-500 and AIME24, a 0.5B LLM outperforms GPT-4o, a 3B LLM
surpasses a 405B LLM, and a 7B LLM beats o1 and DeepSeek-R1, while with higher
inference efficiency. These findings show the significance of adapting TTS
strategies to the specific characteristics of each task and model and indicate
that TTS is a promising approach for enhancing the reasoning abilities of LLMs.
Test-Time Scaling improves Large Language Model performance by optimizing inference computation based on policy models, Process Reward Models, and problem difficulty.
Test-Time Scaling Large Language Models process reward models MATH-500 AIME24 compute-optimal strategy