MathFusion: Enhancing Mathematic Problem-solving of LLM through
  Instruction Fusion
Large Language Models (LLMs) have shown impressive progress in mathematical
reasoning. While data augmentation is promising to enhance mathematical
problem-solving ability, current approaches are predominantly limited to
instance-level modifications-such as rephrasing or generating syntactic
variations-which fail to capture and leverage the intrinsic relational
structures inherent in mathematical knowledge. Inspired by human learning
processes, where mathematical proficiency develops through systematic exposure
to interconnected concepts, we introduce MathFusion, a novel framework that
enhances mathematical reasoning through cross-problem instruction synthesis.
MathFusion implements this through three fusion strategies: (1) sequential
fusion, which chains related problems to model solution dependencies; (2)
parallel fusion, which combines analogous problems to reinforce conceptual
understanding; and (3) conditional fusion, which creates context-aware
selective problems to enhance reasoning flexibility. By applying these
strategies, we generate a new dataset, MathFusionQA, followed by
fine-tuning models (DeepSeekMath-7B, Mistral-7B, Llama3-8B) on it. Experimental
results demonstrate that MathFusion achieves substantial improvements in
mathematical reasoning while maintaining high data efficiency, boosting
performance by 18.0 points in accuracy across diverse benchmarks while
requiring only 45K additional synthetic instructions, representing a
substantial improvement over traditional single-instruction approaches. Our
datasets, models, and code are publicly available at
https://github.com/QizhiPei/mathfusion.
67dcd33626989570158ce8cf
Qizhi Pei
admin_assigned
2025-03-21T13:00:02.682Z
6397f6081323f19c578f142e
https://cdn-avatars.huggingface.co/v1/production/uploads/6397f6081323f19c578f142e/it7FYYKjlLX8wSsMLm8EO.jpeg
QizhiPei
user
QizhiPei
67dcd33626989570158ce8d0
Lijun Wu
admin_assigned
2025-03-21T13:00:09.197Z
643e60d96db6ba8c5ee177ad
/avatars/73ac7740e462ba0b53a2f2480d9f1e3e.svg
Lijun Wu
user
apeters
67dcd33626989570158ce8d1
Zhuoshi Pan
67dcd33626989570158ce8d2
Yu Li
claimed_verified
2025-04-29T08:00:00.327Z
671b852aa4fa4f8f5fb5404c
https://cdn-avatars.huggingface.co/v1/production/uploads/671b852aa4fa4f8f5fb5404c/TDLsgP8WgKW-qaA8Ys-iJ.jpeg
YU LI
user
yu0226
67dcd33626989570158ce8d3
Honglin Lin
admin_assigned
2025-03-21T13:00:16.585Z
640d99628512ec51d7ef71c7
https://cdn-avatars.huggingface.co/v1/production/uploads/640d99628512ec51d7ef71c7/fcBkqnxfxuuuZTqfN_BGy.jpeg
Honglin Lin
user
LHL3341
67dcd33626989570158ce8d4
Chenlin Ming
admin_assigned
2025-03-21T13:00:29.586Z
677e133ee86d0754dc7ce296
/avatars/c16511c1876b50c2d049925c5f320d15.svg
mingchenlin
user
mingchenlin2025
67dcd33626989570158ce8d5
Xin Gao
67dcd33626989570158ce8d6
Conghui He
admin_assigned
2025-03-21T13:00:36.973Z
63f9fca8d4349b157a109eec
/avatars/fa1f2ae7972d7cde99dab178136ccbb0.svg
Conghui He
user
conghui
67dcd33626989570158ce8d7
Rui Yan
MathFusion: Enhancing Mathematic Problem-solving of LLM through
  Instruction Fusion
Large Language Models (LLMs) have shown impressive progress in mathematical
reasoning. While data augmentation is promising to enhance mathematical
problem-solving ability, current approaches are predominantly limited to
instance-level modifications-such as rephrasing or generating syntactic
variations-which fail to capture and leverage the intrinsic relational
structures inherent in mathematical knowledge. Inspired by human learning
processes, where mathematical proficiency develops through systematic exposure
to interconnected concepts, we introduce MathFusion, a novel framework that
enhances mathematical reasoning through cross-problem instruction synthesis.
MathFusion implements this through three fusion strategies: (1) sequential
fusion, which chains related problems to model solution dependencies; (2)
parallel fusion, which combines analogous problems to reinforce conceptual
understanding; and (3) conditional fusion, which creates context-aware
selective problems to enhance reasoning flexibility. By applying these
strategies, we generate a new dataset, MathFusionQA, followed by
fine-tuning models (DeepSeekMath-7B, Mistral-7B, Llama3-8B) on it. Experimental
results demonstrate that MathFusion achieves substantial improvements in
mathematical reasoning while maintaining high data efficiency, boosting
performance by 18.0 points in accuracy across diverse benchmarks while
requiring only 45K additional synthetic instructions, representing a
substantial improvement over traditional single-instruction approaches. Our
datasets, models, and code are publicly available at
https://github.com/QizhiPei/mathfusion.
A new framework, MathFusion, enhances mathematical reasoning by synthesizing cross-problem instructions, leading to significant improvements in accuracy with fewer additional synthetic instructions.
Large Language Models data augmentation mathematical reasoning instance-level modifications intrinsic relational structures systematic exposure interconnected concepts cross-problem instruction synthesis sequential fusion parallel fusion conditional fusion MathFusionQA fine-tuning DeepSeekMath-7B Mistral-7B Llama3-8B data efficiency