Test-Time Scaling in Reasoning Models Is Not Effective for
  Knowledge-Intensive Tasks Yet
Test-time scaling increases inference-time computation by allowing models to
generate long reasoning chains, and has shown strong performance across many
domains. However, in this work, we show that this approach is not yet effective
for knowledge-intensive tasks, where high factual accuracy and low
hallucination rates are essential. We conduct a comprehensive evaluation of
test-time scaling using 12 reasoning models on two knowledge-intensive
benchmarks. Our results reveal that increasing test-time computation does not
consistently improve accuracy and, in many cases, it even leads to more
hallucinations. We then analyze how extended reasoning affects hallucination
behavior. We find that reduced hallucinations often result from the model
choosing to abstain after thinking more, rather than from improved factual
recall. Conversely, for some models, longer reasoning encourages attempts on
previously unanswered questions, many of which result in hallucinations. Case
studies show that extended reasoning can induce confirmation bias, leading to
overconfident hallucinations. Despite these limitations, we observe that
compared to non-thinking, enabling thinking remains beneficial. Code and data
are available at https://github.com/XuZhao0/tts-knowledge
68bfded0207285de11b07df2
James Xu Zhao
68bfded0207285de11b07df3
Bryan Hooi
68bfded0207285de11b07df4
See-Kiong Ng
Test-Time Scaling in Reasoning Models Is Not Effective for
  Knowledge-Intensive Tasks Yet
Test-time scaling increases inference-time computation by allowing models to
generate long reasoning chains, and has shown strong performance across many
domains. However, in this work, we show that this approach is not yet effective
for knowledge-intensive tasks, where high factual accuracy and low
hallucination rates are essential. We conduct a comprehensive evaluation of
test-time scaling using 12 reasoning models on two knowledge-intensive
benchmarks. Our results reveal that increasing test-time computation does not
consistently improve accuracy and, in many cases, it even leads to more
hallucinations. We then analyze how extended reasoning affects hallucination
behavior. We find that reduced hallucinations often result from the model
choosing to abstain after thinking more, rather than from improved factual
recall. Conversely, for some models, longer reasoning encourages attempts on
previously unanswered questions, many of which result in hallucinations. Case
studies show that extended reasoning can induce confirmation bias, leading to
overconfident hallucinations. Despite these limitations, we observe that
compared to non-thinking, enabling thinking remains beneficial. Code and data
are available at https://github.com/XuZhao0/tts-knowledge
Test-time scaling does not consistently improve accuracy or reduce hallucinations in knowledge-intensive tasks, often leading to overconfident errors.
test-time scaling inference-time computation reasoning chains knowledge-intensive tasks factual accuracy hallucination rates reasoning models confirmation bias