QueST: Incentivizing LLMs to Generate Difficult Problems
Large Language Models have achieved strong performance on reasoning tasks,
solving competition-level coding and math problems. However, their scalability
is limited by human-labeled datasets and the lack of large-scale, challenging
coding problem training data. Existing competitive coding datasets contain only
thousands to tens of thousands of problems. Previous synthetic data generation
methods rely on either augmenting existing instruction datasets or selecting
challenging problems from human-labeled data. In this paper, we propose QueST,
a novel framework which combines difficulty-aware graph sampling and
difficulty-aware rejection fine-tuning that directly optimizes specialized
generators to create challenging coding problems. Our trained generators
demonstrate superior capability compared to even GPT-4o at creating challenging
problems that benefit downstream performance. We leverage QueST to generate
large-scale synthetic coding problems, which we then use to distill from strong
teacher models with long chain-of-thought or to conduct reinforcement learning
for smaller models, proving effective in both scenarios. Our distillation
experiments demonstrate significant performance gains. Specifically, after
fine-tuning Qwen3-8B-base on 100K difficult problems generated by QueST, we
surpass the performance of the original Qwen3-8B on LiveCodeBench. With an
additional 112K examples (i.e., 28K human-written problems paired with multiple
synthetic solutions), our 8B model matches the performance of the much larger
DeepSeek-R1-671B. These findings indicate that generating complex problems via
QueST offers an effective and scalable approach to advancing the frontiers of
competitive coding and reasoning for large language models.
68f7545d24c4489363111c55
Hanxu Hu
claimed_verified
2025-10-21T14:00:56.944Z
63898876f7d3b0df09221ed0
/avatars/f7a9fe8a7be30bd07a10df2b286a0711.svg
Hanxu Hu
user
HanxuHU
68f7545d24c4489363111c56
Xingxing Zhang
claimed_verified
2025-10-21T14:01:01.116Z
64181f3d57c3a491a4d6991a
/avatars/8fc0b42c8a71f403dd3b77667841242f.svg
Xingxing Zhang
user
fantasyorg
68f7545d24c4489363111c57
Jannis Vamvas
claimed_verified
2025-10-22T15:45:36.077Z
622a41f973f278bb2fb1c6a4
/avatars/f4945fb277e60cc065ad33c9fcc5be03.svg
Jannis Vamvas
user
jvamvas
68f7545d24c4489363111c58
Rico Sennrich
68f7545d24c4489363111c59
Furu Wei
QueST: Incentivizing LLMs to Generate Difficult Problems
Large Language Models have achieved strong performance on reasoning tasks,
solving competition-level coding and math problems. However, their scalability
is limited by human-labeled datasets and the lack of large-scale, challenging
coding problem training data. Existing competitive coding datasets contain only
thousands to tens of thousands of problems. Previous synthetic data generation
methods rely on either augmenting existing instruction datasets or selecting
challenging problems from human-labeled data. In this paper, we propose QueST,
a novel framework which combines difficulty-aware graph sampling and
difficulty-aware rejection fine-tuning that directly optimizes specialized
generators to create challenging coding problems. Our trained generators
demonstrate superior capability compared to even GPT-4o at creating challenging
problems that benefit downstream performance. We leverage QueST to generate
large-scale synthetic coding problems, which we then use to distill from strong
teacher models with long chain-of-thought or to conduct reinforcement learning
for smaller models, proving effective in both scenarios. Our distillation
experiments demonstrate significant performance gains. Specifically, after
fine-tuning Qwen3-8B-base on 100K difficult problems generated by QueST, we
surpass the performance of the original Qwen3-8B on LiveCodeBench. With an
additional 112K examples (i.e., 28K human-written problems paired with multiple
synthetic solutions), our 8B model matches the performance of the much larger
DeepSeek-R1-671B. These findings indicate that generating complex problems via
QueST offers an effective and scalable approach to advancing the frontiers of
competitive coding and reasoning for large language models.
QueST, a framework combining difficulty-aware graph sampling and fine-tuning, generates large-scale synthetic coding problems to enhance the performance of large language models in competitive coding and reasoning tasks.
difficulty-aware graph sampling difficulty-aware rejection fine-tuning QueST synthetic data generation chain-of-thought reinforcement learning distillation LiveCodeBench DeepSeek-R1-671B