Synthesizing Agentic Data for Web Agents with Progressive Difficulty
  Enhancement Mechanisms
Web-based 'deep research' agents aim to solve complex question - answering
tasks through long-horizon interactions with online tools. These tasks remain
challenging, as the underlying language models are often not optimized for
long-horizon reasoning and exploration. Prior work has proposed workflows for
constructing instruction-tuning datasets, often leveraging knowledge graphs.
However, such methods typically lack fine-grained control over difficulty and
quality, yielding synthetic data that falls short of capturing the complexity
required for long-horizon reasoning. Furthermore, many studies conflate data
and training effects by comparing models trained under different optimization
recipes, making it difficult to isolate and evaluate the effectiveness of the
data itself. We introduce a two-pronged data synthesis pipeline that generates
question - answer pairs by progressively increasing task complexity until a
frontier baseline web agent fails. The baseline agent plays multiple roles in
this process: attempting the questions, validating factuality, checking for
alternative answers, and enforcing filtering. To evaluate the effectiveness of
our synthesis methods, we adopt a controlled training setup based on
distillation from strong web agents. Experiments across multiple web-based
benchmarks show that our dataset - despite being smaller - enables the training
of more effective web agents than existing datasets. In particular, our data
exhibits twice the diversity in tool-use actions, allowing models trained on it
to achieve stronger performance while avoiding repetitive tool-calling
behaviors.
68f2693637ed56c2c926f940
Shrey Pandit
claimed_verified
2025-10-24T16:14:12.037Z
648749094dea003c6dae810f
https://cdn-avatars.huggingface.co/v1/production/uploads/648749094dea003c6dae810f/gHUHSBt1zrt8wjO1YwTNu.jpeg
Shrey Pandit
user
SP2001
68f2693637ed56c2c926f941
Xuan-Phi Nguyen
68f2693637ed56c2c926f942
Yifei Ming
68f2693637ed56c2c926f943
Austin Xu
68f2693637ed56c2c926f944
Jiayu Wang
claimed_verified
2025-10-18T04:47:41.283Z
651651f5d93a51ceda3021c3
https://cdn-avatars.huggingface.co/v1/production/uploads/651651f5d93a51ceda3021c3/FE2uGpTKBRWMKTDBv1H-g.png
Jiayu (Mila) Wang
user
MilaWang
68f2693637ed56c2c926f945
Caiming Xiong
68f2693637ed56c2c926f946
Shafiq Joty
Synthesizing Agentic Data for Web Agents with Progressive Difficulty
  Enhancement Mechanisms
Web-based 'deep research' agents aim to solve complex question - answering
tasks through long-horizon interactions with online tools. These tasks remain
challenging, as the underlying language models are often not optimized for
long-horizon reasoning and exploration. Prior work has proposed workflows for
constructing instruction-tuning datasets, often leveraging knowledge graphs.
However, such methods typically lack fine-grained control over difficulty and
quality, yielding synthetic data that falls short of capturing the complexity
required for long-horizon reasoning. Furthermore, many studies conflate data
and training effects by comparing models trained under different optimization
recipes, making it difficult to isolate and evaluate the effectiveness of the
data itself. We introduce a two-pronged data synthesis pipeline that generates
question - answer pairs by progressively increasing task complexity until a
frontier baseline web agent fails. The baseline agent plays multiple roles in
this process: attempting the questions, validating factuality, checking for
alternative answers, and enforcing filtering. To evaluate the effectiveness of
our synthesis methods, we adopt a controlled training setup based on
distillation from strong web agents. Experiments across multiple web-based
benchmarks show that our dataset - despite being smaller - enables the training
of more effective web agents than existing datasets. In particular, our data
exhibits twice the diversity in tool-use actions, allowing models trained on it
to achieve stronger performance while avoiding repetitive tool-calling
behaviors.
A two-pronged data synthesis pipeline generates complex question-answer pairs, enabling the training of more effective web-based research agents with higher diversity in tool use.
deep research agents long-horizon reasoning instruction-tuning datasets knowledge graphs data synthesis pipeline baseline web agent distillation web-based benchmarks tool-use actions