Open Data Synthesis For Deep Research
Large language models (LLMs) are increasingly expected to go beyond simple
factual queries toward Deep Research-tasks that require decomposing questions
into sub-problems, coordinating multi-step reasoning, and synthesizing evidence
from diverse sources. We formalize Deep Research tasks with verifiable answers
as Hierarchical Constraint Satisfaction Problems (HCSPs), which are
fundamentally different from single-constraint, multi-hop, or flat CSP
formulations. However, existing benchmarks (e.g., Natural Questions, HotpotQA)
fail to capture this complexity, while recent synthetic datasets often
introduce shortcut reasoning, knowledge leakage, or lack sufficient structural
depth. To address this gap, we introduce InfoSeek, a scalable framework for
synthesizing complex Deep Research tasks. InfoSeek uses a dual-agent system to
recursively build a Research Tree from large-scale webpages, blurring
intermediate nodes into valid sub-problems, and converting these trees into
natural language questions that require traversing the full hierarchy. It also
enables rapid scaling, yielding over 50K training examples, a curated test set,
and reasoning trajectories generated via reject sampling. Experiments show that
models trained on InfoSeek consistently outperform strong baselines. On a
challenging benchmark BrowseComp-Plus, 3B LLMs optimized with InfoSeek surpass
much larger 32B models and lightweight commercial APIs (e.g., Gemini2.5-Flash),
while achieving performance comparable to stronger APIs (e.g., Gemini2.5-Pro).
By preserving meta-information such as intermediate steps and retrieval labels,
InfoSeek further supports advanced optimization strategies, including compound
reward design and trajectory-level exploration. We provide our codes and
datasets in https://github.com/VectorSpaceLab/InfoSeek{this repository}.
68b902b5d43cadaf7a688b4c
Ziyi Xia
claimed_verified
2025-09-04T08:43:56.322Z
6540617c7cadb2d1b42007c5
/avatars/b1877fd0564c362a0d4a064d4ec43a73.svg
Ziyi Xia
user
ZiyiXia
68b902b5d43cadaf7a688b4d
Kun Luo
claimed_verified
2025-09-06T10:57:25.689Z
627933c317c069c72a136b35
/avatars/f7aef400e47ac84897e6298c7c881c7a.svg
Kun Luo
user
Lk123
68b902b5d43cadaf7a688b4e
Hongjin Qian
68b902b5d43cadaf7a688b4f
Zheng Liu
Open Data Synthesis For Deep Research
Large language models (LLMs) are increasingly expected to go beyond simple
factual queries toward Deep Research-tasks that require decomposing questions
into sub-problems, coordinating multi-step reasoning, and synthesizing evidence
from diverse sources. We formalize Deep Research tasks with verifiable answers
as Hierarchical Constraint Satisfaction Problems (HCSPs), which are
fundamentally different from single-constraint, multi-hop, or flat CSP
formulations. However, existing benchmarks (e.g., Natural Questions, HotpotQA)
fail to capture this complexity, while recent synthetic datasets often
introduce shortcut reasoning, knowledge leakage, or lack sufficient structural
depth. To address this gap, we introduce InfoSeek, a scalable framework for
synthesizing complex Deep Research tasks. InfoSeek uses a dual-agent system to
recursively build a Research Tree from large-scale webpages, blurring
intermediate nodes into valid sub-problems, and converting these trees into
natural language questions that require traversing the full hierarchy. It also
enables rapid scaling, yielding over 50K training examples, a curated test set,
and reasoning trajectories generated via reject sampling. Experiments show that
models trained on InfoSeek consistently outperform strong baselines. On a
challenging benchmark BrowseComp-Plus, 3B LLMs optimized with InfoSeek surpass
much larger 32B models and lightweight commercial APIs (e.g., Gemini2.5-Flash),
while achieving performance comparable to stronger APIs (e.g., Gemini2.5-Pro).
By preserving meta-information such as intermediate steps and retrieval labels,
InfoSeek further supports advanced optimization strategies, including compound
reward design and trajectory-level exploration. We provide our codes and
datasets in https://github.com/VectorSpaceLab/InfoSeek{this repository}.
InfoSeek is a scalable framework for generating complex Deep Research tasks by synthesizing hierarchical constraint satisfaction problems, enabling models to outperform larger baselines on challenging benchmarks.
Hierarchical Constraint Satisfaction Problems HCSPs dual-agent system Research Tree natural language questions reasoning trajectories reject sampling compound reward design trajectory-level exploration