Does More Inference-Time Compute Really Help Robustness?
Recently, Zaremba et al. demonstrated that increasing inference-time
computation improves robustness in large proprietary reasoning LLMs. In this
paper, we first show that smaller-scale, open-source models (e.g., DeepSeek R1,
Qwen3, Phi-reasoning) can also benefit from inference-time scaling using a
simple budget forcing strategy. More importantly, we reveal and critically
examine an implicit assumption in prior work: intermediate reasoning steps are
hidden from adversaries. By relaxing this assumption, we identify an important
security risk, intuitively motivated and empirically verified as an inverse
scaling law: if intermediate reasoning steps become explicitly accessible,
increased inference-time computation consistently reduces model robustness.
Finally, we discuss practical scenarios where models with hidden reasoning
chains are still vulnerable to attacks, such as models with tool-integrated
reasoning and advanced reasoning extraction attacks. Our findings collectively
demonstrate that the robustness benefits of inference-time scaling depend
heavily on the adversarial setting and deployment context. We urge
practitioners to carefully weigh these subtle trade-offs before applying
inference-time scaling in security-sensitive, real-world applications.
688061c500f4b5a05f2fbe89
Tong Wu
admin_assigned
2025-07-23T13:32:53.276Z
62fae9328e137d7c4b896498
/avatars/1bda39dec585c099417cc9daa9f53c42.svg
Tong Wu
user
tongwu2020
688061c500f4b5a05f2fbe8a
Chong Xiang
688061c500f4b5a05f2fbe8b
Jiachen T. Wang
688061c500f4b5a05f2fbe8c
Weichen Yu
688061c500f4b5a05f2fbe8d
Chawin Sitawarin
688061c500f4b5a05f2fbe8e
Vikash Sehwag
688061c500f4b5a05f2fbe8f
Prateek Mittal
Does More Inference-Time Compute Really Help Robustness?
Recently, Zaremba et al. demonstrated that increasing inference-time
computation improves robustness in large proprietary reasoning LLMs. In this
paper, we first show that smaller-scale, open-source models (e.g., DeepSeek R1,
Qwen3, Phi-reasoning) can also benefit from inference-time scaling using a
simple budget forcing strategy. More importantly, we reveal and critically
examine an implicit assumption in prior work: intermediate reasoning steps are
hidden from adversaries. By relaxing this assumption, we identify an important
security risk, intuitively motivated and empirically verified as an inverse
scaling law: if intermediate reasoning steps become explicitly accessible,
increased inference-time computation consistently reduces model robustness.
Finally, we discuss practical scenarios where models with hidden reasoning
chains are still vulnerable to attacks, such as models with tool-integrated
reasoning and advanced reasoning extraction attacks. Our findings collectively
demonstrate that the robustness benefits of inference-time scaling depend
heavily on the adversarial setting and deployment context. We urge
practitioners to carefully weigh these subtle trade-offs before applying
inference-time scaling in security-sensitive, real-world applications.
Inference-time scaling improves robustness in large models but can reduce robustness in smaller models if intermediate reasoning steps are exposed, highlighting the need for careful consideration in security-sensitive applications.
inference-time scaling robustness LLMs DeepSeek R1 Qwen3 Phi-reasoning budget forcing strategy intermediate reasoning steps adversarial setting deployment context tool-integrated reasoning reasoning extraction attacks