Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis
  from Scratch
The availability of high-quality data is one of the most important factors in
improving the reasoning capability of LLMs. Existing works have demonstrated
the effectiveness of creating more instruction data from seed questions or
knowledge bases. Recent research indicates that continually scaling up data
synthesis from strong models (e.g., GPT-4) can further elicit reasoning
performance. Though promising, the open-sourced community still lacks
high-quality data at scale and scalable data synthesis methods with affordable
costs. To address this, we introduce ScaleQuest, a scalable and novel data
synthesis method that utilizes "small-size" (e.g., 7B) open-source models to
generate questions from scratch without the need for seed data with complex
augmentation constraints. With the efficient ScaleQuest, we automatically
constructed a mathematical reasoning dataset consisting of 1 million
problem-solution pairs, which are more effective than existing open-sourced
datasets. It can universally increase the performance of mainstream open-source
models (i.e., Mistral, Llama3, DeepSeekMath, and Qwen2-Math) by achieving 29.2%
to 46.4% gains on MATH. Notably, simply fine-tuning the Qwen2-Math-7B-Base
model with our dataset can even surpass Qwen2-Math-7B-Instruct, a strong and
well-aligned model on closed-source data, and proprietary models such as
GPT-4-Turbo and Claude-3.5 Sonnet.
671b0c864e0ac814c30b16f1
Yuyang Ding
claimed_verified
2024-10-25T09:29:45.039Z
626cf0f65651e31a7a2b9779
https://cdn-avatars.huggingface.co/v1/production/uploads/626cf0f65651e31a7a2b9779/TAE--QIKo1vz4Rb8aearl.jpeg
Ding
user
dyyyyyyyy
671b0c864e0ac814c30b16f2
Xinyu Shi
admin_assigned
2024-10-25T11:25:46.041Z
6380ff85f496d57325c2ac2b
/avatars/0b9eb9c8a96d5ffd905f9ddeee78e6f3.svg
Xinyu Shi
user
XinyuShi
671b0c864e0ac814c30b16f3
Xiaobo Liang
claimed_verified
2024-10-30T09:54:48.587Z
64116770230ce11b1be7f3b2
/avatars/d554406a121666c3e16bc3321415ccb1.svg
Xiaobo Liang
user
dropreg
671b0c864e0ac814c30b16f4
Juntao Li
admin_assigned
2024-10-25T11:26:05.777Z
6670e285b0c03c4e9d6e0985
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/uCZHm4gKSHZ2b0hpHWgZv.jpeg
Juntao Li
user
douvleplus
671b0c864e0ac814c30b16f5
Qiaoming Zhu
671b0c864e0ac814c30b16f6
Min Zhang
Unleashing Reasoning Capability of LLMs via Scalable Question Synthesis
  from Scratch
The availability of high-quality data is one of the most important factors in
improving the reasoning capability of LLMs. Existing works have demonstrated
the effectiveness of creating more instruction data from seed questions or
knowledge bases. Recent research indicates that continually scaling up data
synthesis from strong models (e.g., GPT-4) can further elicit reasoning
performance. Though promising, the open-sourced community still lacks
high-quality data at scale and scalable data synthesis methods with affordable
costs. To address this, we introduce ScaleQuest, a scalable and novel data
synthesis method that utilizes "small-size" (e.g., 7B) open-source models to
generate questions from scratch without the need for seed data with complex
augmentation constraints. With the efficient ScaleQuest, we automatically
constructed a mathematical reasoning dataset consisting of 1 million
problem-solution pairs, which are more effective than existing open-sourced
datasets. It can universally increase the performance of mainstream open-source
models (i.e., Mistral, Llama3, DeepSeekMath, and Qwen2-Math) by achieving 29.2%
to 46.4% gains on MATH. Notably, simply fine-tuning the Qwen2-Math-7B-Base
model with our dataset can even surpass Qwen2-Math-7B-Instruct, a strong and
well-aligned model on closed-source data, and proprietary models such as
GPT-4-Turbo and Claude-3.5 Sonnet.
ScaleQuest is a scalable data synthesis method that generates high-quality mathematical reasoning datasets, improving the performance of open-source models.
LLMs data synthesis instruction data GPT-4 scalable data synthesis open-source models SeedQuest small-size models mathematical reasoning dataset problem-solution pairs Mistral Llama3 DeepSeekMath Qwen2-Math MATH fine-tuning