Making Mathematical Reasoning Adaptive
Mathematical reasoning is a primary indicator of large language models (LLMs)
intelligence. However, existing LLMs exhibit failures of robustness and
generalization. This paper attributes these deficiencies to spurious reasoning,
i.e., producing answers from superficial features. To address this challenge,
we propose the AdaR framework to enable adaptive reasoning, wherein models rely
on problem-solving logic to produce answers. AdaR synthesizes logically
equivalent queries by varying variable values, and trains models with RLVR on
these data to penalize spurious logic while encouraging adaptive logic. To
improve data quality, we extract the problem-solving logic from the original
query and generate the corresponding answer by code execution, then apply a
sanity check. Experimental results demonstrate that AdaR improves robustness
and generalization, achieving substantial improvement in mathematical reasoning
while maintaining high data efficiency. Analysis indicates that data synthesis
and RLVR function in a coordinated manner to enable adaptive reasoning in LLMs.
Subsequent analyses derive key design insights into the effect of critical
factors and the applicability to instruct LLMs. Our project is available at
https://github.com/LaiZhejian/AdaR
68e5159bf9af2f6567eab906
Zhejian Lai
claimed_verified
2025-10-08T08:01:46.085Z
643525ea0b30bd434ea15363
https://cdn-avatars.huggingface.co/v1/production/uploads/643525ea0b30bd434ea15363/7sAzllfWUPtt68NY1gDLj.png
Jackie Lai
user
DreamW1ngs
68e5159bf9af2f6567eab907
Xiang Geng
68e5159bf9af2f6567eab908
Zhijun Wang
68e5159bf9af2f6567eab909
Yang Bai
68e5159bf9af2f6567eab90a
Jiahuan Li
68e5159bf9af2f6567eab90b
Rongxiang Weng
68e5159bf9af2f6567eab90c
Jingang Wang
68e5159bf9af2f6567eab90d
Xuezhi Cao
68e5159bf9af2f6567eab90e
Xunliang Cai
68e5159bf9af2f6567eab90f
Shujian Huang
Making Mathematical Reasoning Adaptive
Mathematical reasoning is a primary indicator of large language models (LLMs)
intelligence. However, existing LLMs exhibit failures of robustness and
generalization. This paper attributes these deficiencies to spurious reasoning,
i.e., producing answers from superficial features. To address this challenge,
we propose the AdaR framework to enable adaptive reasoning, wherein models rely
on problem-solving logic to produce answers. AdaR synthesizes logically
equivalent queries by varying variable values, and trains models with RLVR on
these data to penalize spurious logic while encouraging adaptive logic. To
improve data quality, we extract the problem-solving logic from the original
query and generate the corresponding answer by code execution, then apply a
sanity check. Experimental results demonstrate that AdaR improves robustness
and generalization, achieving substantial improvement in mathematical reasoning
while maintaining high data efficiency. Analysis indicates that data synthesis
and RLVR function in a coordinated manner to enable adaptive reasoning in LLMs.
Subsequent analyses derive key design insights into the effect of critical
factors and the applicability to instruct LLMs. Our project is available at
https://github.com/LaiZhejian/AdaR
AdaR framework enhances LLMs' robustness and generalization in mathematical reasoning by synthesizing logically equivalent queries and using RLVR to penalize spurious logic.
AdaR adaptive reasoning spurious reasoning logically equivalent queries RLVR data synthesis mathematical reasoning data efficiency