Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution
Recent breakthroughs in large language models (LLMs) on reasoning tasks rely
heavily on massive, high-quality datasets-typically human-annotated and thus
difficult to scale. While data synthesis or distillation offers a promising
alternative, existing methods struggle with inconsistent data quality and an
inability to dynamically adapt to the evolving capabilities of the model,
leading to suboptimal training signals. To address these limitations, we
introduce Socratic-Zero, a fully autonomous framework that generates
high-quality training data from minimal seed examples through the co-evolution
of three agents: the Teacher, the Solver, and the Generator. The Solver
continuously refines its reasoning by learning from preference feedback on both
successful and failed trajectories; the Teacher adaptively crafts increasingly
challenging questions based on the Solver's weaknesses; and the Generator
distills the Teacher's question-design strategy to enable scalable,
high-fidelity curriculum generation. This closed-loop system produces a
self-improving curriculum-requiring no pre-existing tasks or labels.
Remarkably, starting from only 100 seed questions, our Socratic-Solver-8B
achieves an average gain of +20.2 percentage points over prior data synthesis
methods across seven mathematical reasoning benchmarks (AMC23, AIME24-25,
Olympiad, MATH-500, Minerva, and GSM8K), with consistent gains on both Qwen3
and GLM4 series models. Even more surprisingly, synthetic data from
Socratic-Generator-32B enables student LLMs to achieve superior performance
compared to other state-of-the-art (SOTA) commercial LLMs on these benchmarks,
including Qwen3-235B-A22B, DeepSeek-V3.1-671B, GPT-5, Gemini-2.5-Pro, Grok-4,
and Claude-4.1-Opus.
68dbf4824159d1f2418f9802
Shaobo Wang
claimed_verified
2025-10-01T10:23:49.442Z
66968099c952e09a4cb29f78
https://cdn-avatars.huggingface.co/v1/production/uploads/66968099c952e09a4cb29f78/n90NI2R3E9_RqCyMjDCQF.webp
Wang
user
Steven-Shaobo
68dbf4824159d1f2418f9803
Zhengbo Jiao
68dbf4824159d1f2418f9804
Zifan Zhang
68dbf4824159d1f2418f9805
Yilang Peng
68dbf4824159d1f2418f9806
Xu Ze
68dbf4824159d1f2418f9807
Boyu Yang
68dbf4824159d1f2418f9808
Wei Wang
68dbf4824159d1f2418f9809
Hu Wei
68dbf4824159d1f2418f980a
Linfeng Zhang
Socratic-Zero : Bootstrapping Reasoning via Data-Free Agent Co-evolution
Recent breakthroughs in large language models (LLMs) on reasoning tasks rely
heavily on massive, high-quality datasets-typically human-annotated and thus
difficult to scale. While data synthesis or distillation offers a promising
alternative, existing methods struggle with inconsistent data quality and an
inability to dynamically adapt to the evolving capabilities of the model,
leading to suboptimal training signals. To address these limitations, we
introduce Socratic-Zero, a fully autonomous framework that generates
high-quality training data from minimal seed examples through the co-evolution
of three agents: the Teacher, the Solver, and the Generator. The Solver
continuously refines its reasoning by learning from preference feedback on both
successful and failed trajectories; the Teacher adaptively crafts increasingly
challenging questions based on the Solver's weaknesses; and the Generator
distills the Teacher's question-design strategy to enable scalable,
high-fidelity curriculum generation. This closed-loop system produces a
self-improving curriculum-requiring no pre-existing tasks or labels.
Remarkably, starting from only 100 seed questions, our Socratic-Solver-8B
achieves an average gain of +20.2 percentage points over prior data synthesis
methods across seven mathematical reasoning benchmarks (AMC23, AIME24-25,
Olympiad, MATH-500, Minerva, and GSM8K), with consistent gains on both Qwen3
and GLM4 series models. Even more surprisingly, synthetic data from
Socratic-Generator-32B enables student LLMs to achieve superior performance
compared to other state-of-the-art (SOTA) commercial LLMs on these benchmarks,
including Qwen3-235B-A22B, DeepSeek-V3.1-671B, GPT-5, Gemini-2.5-Pro, Grok-4,
and Claude-4.1-Opus.
A framework called Socratic-Zero autonomously generates high-quality training data through the co-evolution of three agents, improving reasoning tasks in large language models.
large language models LLMs reasoning tasks data synthesis distillation curriculum generation Teacher Solver Generator preference feedback trajectories question-design strategy self-improving curriculum mathematical reasoning benchmarks Socratic-Solver-8B Socratic-Generator-32B Qwen3 GLM4 Qwen3-235B-A22B DeepSeek-V3.1-671B GPT-5 Gemini-2.5-Pro Grok-4 Claude-4.1-Opus