Mathematical Reasoning Planning for Language Models
Bao Nguyen, Hieu Nguyen, Ruifeng She, Xiaojin Fu, Viet Anh Nguyen
Selecting an appropriate reasoning method for a given query remains a key challenge in language model generation. Existing methods typically generate multiple candidate answers and use an aggregation strategy to select the output answer, often assuming that more candidate answers yield higher accuracy. We challenge this assumption through a rigorous theoretical analysis, deriving accuracy bounds for common aggregation methods under fixed generation distributions and candidate sizes.  Building on these insights, we introduce EPIC, an $\textbf{E}$nsemble $\textbf{P}$lann$\textbf{I}$ng with $\textbf{C}$ontrastive learning framework to learn a shared representation space that captures both model reasoning abilities and query-method compatibility. EPIC incorporates our probability bounds as a regularizer in a utility-driven optimization that balances accuracy and computational cost. Experiments on diverse mathematical reasoning tasks show that EPIC consistently selects optimal reasoning methods, improving accuracy while reducing computational overhead.