A Rigorous Benchmark with Multidimensional Evaluation for Deep Research
  Agents: From Answers to Reports
Artificial intelligence is undergoing the paradigm shift from closed language
models to interconnected agent systems capable of external perception and
information integration. As a representative embodiment, Deep Research Agents
(DRAs) systematically exhibit the capabilities for task decomposition,
cross-source retrieval, multi-stage reasoning, and structured output, which
markedly enhance performance on complex and open-ended tasks. However, existing
benchmarks remain deficient in evaluation dimensions, response formatting, and
scoring mechanisms, limiting their capacity to assess such systems effectively.
This paper introduces a rigorous benchmark and a multidimensional evaluation
framework tailored to DRAs and report-style responses. The benchmark comprises
214 expert-curated challenging queries distributed across 10 broad thematic
domains, each accompanied by manually constructed reference bundles to support
composite evaluation. The framework enables comprehensive evaluation of
long-form reports generated by DRAs, incorporating integrated scoring metrics
for semantic quality, topical focus, and retrieval trustworthiness. Extensive
experimentation confirms the superior performance of mainstream DRAs over
web-search-tool-augmented reasoning models, yet reveals considerable scope for
further improvement. This study provides a robust foundation for capability
assessment, architectural refinement, and paradigm advancement in DRA systems.
68df3194df49fb0df1e03c4e
Yang Yao
claimed_verified
2025-10-09T06:52:15.818Z
68e673f5f7b5b87f952b8568
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/RvG3sSBXmWjVuGNwcvDGT.png
YANG YAO
user
EVIGBYEN
68df3194df49fb0df1e03c4f
Yixu Wang
claimed_verified
2025-10-09T12:49:18.138Z
64576c37900b28f784111822
/avatars/88aae11cb2bd7f138b266d82a951508d.svg
yxwang
user
yxwang
68df3194df49fb0df1e03c50
Yuxuan Zhang
claimed_verified
2025-10-05T12:47:11.461Z
63b908d0e3c78740d8e950d0
/avatars/3e80075e92aebdfea712f70b00d5ec7d.svg
Yuxuan Zhang
user
Reacherx
68df3194df49fb0df1e03c51
Yi Lu
claimed_verified
2025-10-05T12:47:15.969Z
65095591cc02352e1e106d0a
https://cdn-avatars.huggingface.co/v1/production/uploads/65095591cc02352e1e106d0a/QgNhKBItxxHY927pBVBai.jpeg
Tom Lu
user
eigentom
68df3194df49fb0df1e03c52
Tianle Gu
68df3194df49fb0df1e03c53
Lingyu Li
68df3194df49fb0df1e03c54
Dingyi Zhao
68df3194df49fb0df1e03c55
Keming Wu
claimed_verified
2025-10-11T14:05:46.529Z
66bf00ca5b4e241fe266059d
https://cdn-avatars.huggingface.co/v1/production/uploads/66bf00ca5b4e241fe266059d/UPjEAg5fHAeC44QvsJcwX.jpeg
Keming Wu
user
wukeming11
68df3194df49fb0df1e03c56
Haozhe Wang
68df3194df49fb0df1e03c57
Ping Nie
claimed_verified
2025-10-05T12:47:13.768Z
65358802a920f38780b3248a
/avatars/9415510b598079973c2b0436ad12db9c.svg
Ping Nie
user
pingnieuk
68df3194df49fb0df1e03c58
Yan Teng
68df3194df49fb0df1e03c59
Yingchun Wang
A Rigorous Benchmark with Multidimensional Evaluation for Deep Research
  Agents: From Answers to Reports
Artificial intelligence is undergoing the paradigm shift from closed language
models to interconnected agent systems capable of external perception and
information integration. As a representative embodiment, Deep Research Agents
(DRAs) systematically exhibit the capabilities for task decomposition,
cross-source retrieval, multi-stage reasoning, and structured output, which
markedly enhance performance on complex and open-ended tasks. However, existing
benchmarks remain deficient in evaluation dimensions, response formatting, and
scoring mechanisms, limiting their capacity to assess such systems effectively.
This paper introduces a rigorous benchmark and a multidimensional evaluation
framework tailored to DRAs and report-style responses. The benchmark comprises
214 expert-curated challenging queries distributed across 10 broad thematic
domains, each accompanied by manually constructed reference bundles to support
composite evaluation. The framework enables comprehensive evaluation of
long-form reports generated by DRAs, incorporating integrated scoring metrics
for semantic quality, topical focus, and retrieval trustworthiness. Extensive
experimentation confirms the superior performance of mainstream DRAs over
web-search-tool-augmented reasoning models, yet reveals considerable scope for
further improvement. This study provides a robust foundation for capability
assessment, architectural refinement, and paradigm advancement in DRA systems.
A benchmark and evaluation framework for Deep Research Agents (DRAs) assesses their performance on complex tasks with multidimensional metrics.
Deep Research Agents task decomposition cross-source retrieval multi-stage reasoning structured output benchmark evaluation framework thematic domains reference bundles semantic quality topical focus retrieval trustworthiness