When To Solve, When To Verify: Compute-Optimal Problem Solving and
  Generative Verification for LLM Reasoning
Scaling test-time compute has emerged as a key strategy for enhancing the
reasoning capabilities of large language models (LLMs), particularly in tasks
like mathematical problem-solving. A traditional approach, Self-Consistency
(SC), generates multiple solutions to a problem and selects the most common
answer via majority voting. Another common method involves scoring each
solution with a reward model (verifier) and choosing the best one. Recent
advancements in Generative Reward Models (GenRM) reframe verification as a
next-token prediction task, enabling inference-time scaling along a new axis.
Specifically, GenRM generates multiple verification chains-of-thought to score
each solution. Under a limited inference budget, this introduces a fundamental
trade-off: should you spend the budget on scaling solutions via SC or generate
fewer solutions and allocate compute to verification via GenRM? To address
this, we evaluate GenRM against SC under a fixed inference budget.
Interestingly, we find that SC is more compute-efficient than GenRM for most
practical inference budgets across diverse models and datasets. For instance,
GenRM first matches SC after consuming up to 8x the inference compute and
requires significantly more compute to outperform it. Furthermore, we derive
inference scaling laws for the GenRM paradigm, revealing that compute-optimal
inference favors scaling solution generation more aggressively than scaling the
number of verifications. Our work provides practical guidance on optimizing
test-time scaling by balancing solution generation and verification. The code
is available at https://github.com/nishadsinghi/sc-genrm-scaling.
67ec95ea3d267d26663ea34b
Nishad Singhi
claimed_verified
2025-04-02T13:56:07.949Z
641f72d8eefe94aff6e21068
/avatars/0f07cffb49698232c4bd912898abc920.svg
Nishad Singhi
user
nishadsinghi
67ec95ea3d267d26663ea34c
Hritik Bansal
admin_assigned
2025-04-02T11:16:19.164Z
61c5c25705aa54027c52f7b3
/avatars/8a89e040dc331b7a83d9a704c4fc29d2.svg
Hritik Bansal
user
hbXNov
67ec95ea3d267d26663ea34d
Arian Hosseini
admin_assigned
2025-04-02T11:17:10.400Z
62e447e9640206c5d21e5ff6
/avatars/42d8247c2ffa98a4c6255ce84051f1b9.svg
Arian Hosseini
user
arianhosseini
67ec95ea3d267d26663ea34e
Aditya Grover
admin_assigned
2025-04-02T11:17:22.994Z
65d38c1e5d5749637491679b
/avatars/010a6588f382d6d4394216549a2385e3.svg
Aditya Grover
user
adityagrover
67ec95ea3d267d26663ea34f
Kai-Wei Chang
admin_assigned
2025-04-02T11:17:29.569Z
60b7b9d71b90c5d07c23fbd0
https://cdn-avatars.huggingface.co/v1/production/uploads/1622653364258-noauth.jpeg
Kai-Wei Chang
user
kaiweichang
67ec95ea3d267d26663ea350
Marcus Rohrbach
67ec95ea3d267d26663ea351
Anna Rohrbach
admin_assigned
2025-04-02T11:16:35.830Z
6397ca710bdcc7e3216f3c49
/avatars/51ad23a000eaf817c2e47149810b0156.svg
Anna Rohrbach
user
anna-rohrbach
When To Solve, When To Verify: Compute-Optimal Problem Solving and
  Generative Verification for LLM Reasoning
Scaling test-time compute has emerged as a key strategy for enhancing the
reasoning capabilities of large language models (LLMs), particularly in tasks
like mathematical problem-solving. A traditional approach, Self-Consistency
(SC), generates multiple solutions to a problem and selects the most common
answer via majority voting. Another common method involves scoring each
solution with a reward model (verifier) and choosing the best one. Recent
advancements in Generative Reward Models (GenRM) reframe verification as a
next-token prediction task, enabling inference-time scaling along a new axis.
Specifically, GenRM generates multiple verification chains-of-thought to score
each solution. Under a limited inference budget, this introduces a fundamental
trade-off: should you spend the budget on scaling solutions via SC or generate
fewer solutions and allocate compute to verification via GenRM? To address
this, we evaluate GenRM against SC under a fixed inference budget.
Interestingly, we find that SC is more compute-efficient than GenRM for most
practical inference budgets across diverse models and datasets. For instance,
GenRM first matches SC after consuming up to 8x the inference compute and
requires significantly more compute to outperform it. Furthermore, we derive
inference scaling laws for the GenRM paradigm, revealing that compute-optimal
inference favors scaling solution generation more aggressively than scaling the
number of verifications. Our work provides practical guidance on optimizing
test-time scaling by balancing solution generation and verification. The code
is available at https://github.com/nishadsinghi/sc-genrm-scaling.
Self-Consistency is more compute-efficient than Generative Reward Models for most practical inference budgets in large language models.
Self-Consistency Generative Reward Models next-token prediction chains-of-thought inference scaling laws