WideSearch: Benchmarking Agentic Broad Info-Seeking
From professional research to everyday planning, many tasks are bottlenecked
by wide-scale information seeking, which is more repetitive than cognitively
complex. With the rapid development of Large Language Models (LLMs), automated
search agents powered by LLMs offer a promising solution to liberate humans
from this tedious work. However, the capability of these agents to perform such
"wide-context" collection reliably and completely remains largely unevaluated
due to a lack of suitable benchmarks. To bridge this gap, we introduce
WideSearch, a new benchmark engineered to evaluate agent reliability on these
large-scale collection tasks. The benchmark features 200 manually curated
questions (100 in English, 100 in Chinese) from over 15 diverse domains,
grounded in real user queries. Each task requires agents to collect large-scale
atomic information, which could be verified one by one objectively, and arrange
it into a well-organized output. A rigorous five-stage quality control pipeline
ensures the difficulty, completeness, and verifiability of the dataset. We
benchmark over 10 state-of-the-art agentic search systems, including
single-agent, multi-agent frameworks, and end-to-end commercial systems. Most
systems achieve overall success rates near 0\%, with the best performer
reaching just 5\%. However, given sufficient time, cross-validation by multiple
human testers can achieve a near 100\% success rate. These results demonstrate
that present search agents have critical deficiencies in large-scale
information seeking, underscoring urgent areas for future research and
development in agentic search. Our dataset, evaluation pipeline, and benchmark
results have been publicly released at https://widesearch-seed.github.io/
689aa9c6fab6fdd2e52ac459
Ryan Wong
689aa9c6fab6fdd2e52ac45a
Jiawei Wang
claimed_verified
2025-08-13T07:20:54.657Z
64060b49a577649430bf6974
/avatars/74d0d6ed656b593e4c101b09edf18c7a.svg
Jiawei Wang
user
Jarvis1111
689aa9c6fab6fdd2e52ac45b
Junjie Zhao
689aa9c6fab6fdd2e52ac45c
Li Chen
689aa9c6fab6fdd2e52ac45d
Yan Gao
689aa9c6fab6fdd2e52ac45e
Long Zhang
689aa9c6fab6fdd2e52ac45f
Xuan Zhou
689aa9c6fab6fdd2e52ac460
Zuo Wang
689aa9c6fab6fdd2e52ac461
Kai Xiang
689aa9c6fab6fdd2e52ac462
Ge Zhang
claimed_verified
2025-08-13T07:20:43.641Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
689aa9c6fab6fdd2e52ac463
Wenhao Huang
689aa9c6fab6fdd2e52ac464
Yang Wang
689aa9c6fab6fdd2e52ac465
Ke Wang
WideSearch: Benchmarking Agentic Broad Info-Seeking
From professional research to everyday planning, many tasks are bottlenecked
by wide-scale information seeking, which is more repetitive than cognitively
complex. With the rapid development of Large Language Models (LLMs), automated
search agents powered by LLMs offer a promising solution to liberate humans
from this tedious work. However, the capability of these agents to perform such
"wide-context" collection reliably and completely remains largely unevaluated
due to a lack of suitable benchmarks. To bridge this gap, we introduce
WideSearch, a new benchmark engineered to evaluate agent reliability on these
large-scale collection tasks. The benchmark features 200 manually curated
questions (100 in English, 100 in Chinese) from over 15 diverse domains,
grounded in real user queries. Each task requires agents to collect large-scale
atomic information, which could be verified one by one objectively, and arrange
it into a well-organized output. A rigorous five-stage quality control pipeline
ensures the difficulty, completeness, and verifiability of the dataset. We
benchmark over 10 state-of-the-art agentic search systems, including
single-agent, multi-agent frameworks, and end-to-end commercial systems. Most
systems achieve overall success rates near 0\%, with the best performer
reaching just 5\%. However, given sufficient time, cross-validation by multiple
human testers can achieve a near 100\% success rate. These results demonstrate
that present search agents have critical deficiencies in large-scale
information seeking, underscoring urgent areas for future research and
development in agentic search. Our dataset, evaluation pipeline, and benchmark
results have been publicly released at https://widesearch-seed.github.io/
WideSearch is a new benchmark evaluating the reliability of automated search agents in large-scale information collection tasks, revealing significant deficiencies in current systems.
Large Language Models automated search agents WideSearch benchmark quality control pipeline agentic search systems single-agent multi-agent frameworks end-to-end commercial systems