arxiv:2510.04617

Making Mathematical Reasoning Adaptive

Published on Oct 6

· Submitted by

Jackie Lai on Oct 14

Upvote

Authors:

Zhejian Lai ,

Abstract

AdaR framework enhances LLMs' robustness and generalization in mathematical reasoning by synthesizing logically equivalent queries and using RLVR to penalize spurious logic.

AI-generated summary

Mathematical reasoning is a primary indicator of large language models (LLMs) intelligence. However, existing LLMs exhibit failures of robustness and generalization. This paper attributes these deficiencies to spurious reasoning, i.e., producing answers from superficial features. To address this challenge, we propose the AdaR framework to enable adaptive reasoning, wherein models rely on problem-solving logic to produce answers. AdaR synthesizes logically equivalent queries by varying variable values, and trains models with RLVR on these data to penalize spurious logic while encouraging adaptive logic. To improve data quality, we extract the problem-solving logic from the original query and generate the corresponding answer by code execution, then apply a sanity check. Experimental results demonstrate that AdaR improves robustness and generalization, achieving substantial improvement in mathematical reasoning while maintaining high data efficiency. Analysis indicates that data synthesis and RLVR function in a coordinated manner to enable adaptive reasoning in LLMs. Subsequent analyses derive key design insights into the effect of critical factors and the applicability to instruct LLMs. Our project is available at https://github.com/LaiZhejian/AdaR

View arXiv page View PDF GitHub 12 Add to collection

Community

DreamW1ngs

Paper author Paper submitter 16 days ago

🌱 Overview

Large Language Models (LLMs) have shown impressive reasoning capabilities, yet they often rely on spurious reasoning — producing answers from superficial features, leading to failure at robustness and generalization.

We propose AdaR framework to enable adaptive reasoning, wherein models rely on problem-solving logic to produce answers. AdaR synthesizes logically equivalent queries by varying variable values, and trains models with RLVR on these data to penalize spurious logic while encouraging adaptive logic.

The framework integrates data synthesis and RLVR training to enhance both robustness (in-domain) and generalization (out-of-domain).

Figure 1.
Subfigure I: Three reasoning modes — direct inference (black), spurious reasoning (red), adaptive reasoning (green).
Subfigure II: Logic-preserving variable perturbation and gold-answer generation via executable logic.
Subfigure III: RLVR optimization encouraging adaptive reasoning through comparative feedback.

📈 Highlights

🚀 +8.5 Average Improvement across in-domain robustness tasks and out-of-domain tasks.
🧮 Only 9K synthetic data needed for significant gains.
⚖️ Enable algebraic thinking and improved stability under scaling.
🔁 Generalizable framework applicable to instruct models.