DeepResearch Arena: The First Exam of LLMs' Research Abilities via
  Seminar-Grounded Tasks
Deep research agents have attracted growing attention for their potential to
orchestrate multi-stage research workflows, spanning literature synthesis,
methodological design, and empirical verification. Despite these strides,
evaluating their research capability faithfully is rather challenging due to
the difficulty of collecting frontier research questions that genuinely capture
researchers' attention and intellectual curiosity. To address this gap, we
introduce DeepResearch Arena, a benchmark grounded in academic seminars that
capture rich expert discourse and interaction, better reflecting real-world
research environments and reducing the risk of data leakage. To automatically
construct DeepResearch Arena, we propose a Multi-Agent Hierarchical Task
Generation (MAHTG) system that extracts research-worthy inspirations from
seminar transcripts. The MAHTG system further translates research-worthy
inspirations into high-quality research tasks, ensuring the traceability of
research task formulation while filtering noise. With the MAHTG system, we
curate DeepResearch Arena with over 10,000 high-quality research tasks from
over 200 academic seminars, spanning 12 disciplines, such as literature,
history, and science. Our extensive evaluation shows that DeepResearch Arena
presents substantial challenges for current state-of-the-art agents, with clear
performance gaps observed across different models.
68b98e69736018af705e8d41
Haiyuan Wan
claimed_verified
2025-09-04T19:36:56.793Z
65a8bcb717d869bb7487c2a1
/avatars/261c28f7e616a8482970f50c1f8919fd.svg
Haiyuan Wan
user
haiyuanwan
68b98e69736018af705e8d42
Chen Yang
68b98e69736018af705e8d43
Junchi Yu
68b98e69736018af705e8d44
Meiqi Tu
68b98e69736018af705e8d45
Jiaxuan Lu
68b98e69736018af705e8d46
Di Yu
68b98e69736018af705e8d47
Jianbao Cao
68b98e69736018af705e8d48
Ben Gao
68b98e69736018af705e8d49
Jiaqing Xie
68b98e69736018af705e8d4a
Aoran Wang
claimed_verified
2025-09-05T14:16:19.262Z
666839475ddf78d3bc5b80f4
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/j4OBskJ_GMS2QGE_nEuEJ.jpeg
Aoran Wang
user
RalfWang
68b98e69736018af705e8d4b
Wenlong Zhang
68b98e69736018af705e8d4c
Philip Torr
68b98e69736018af705e8d4d
Dongzhan Zhou
DeepResearch Arena: The First Exam of LLMs' Research Abilities via
  Seminar-Grounded Tasks
Deep research agents have attracted growing attention for their potential to
orchestrate multi-stage research workflows, spanning literature synthesis,
methodological design, and empirical verification. Despite these strides,
evaluating their research capability faithfully is rather challenging due to
the difficulty of collecting frontier research questions that genuinely capture
researchers' attention and intellectual curiosity. To address this gap, we
introduce DeepResearch Arena, a benchmark grounded in academic seminars that
capture rich expert discourse and interaction, better reflecting real-world
research environments and reducing the risk of data leakage. To automatically
construct DeepResearch Arena, we propose a Multi-Agent Hierarchical Task
Generation (MAHTG) system that extracts research-worthy inspirations from
seminar transcripts. The MAHTG system further translates research-worthy
inspirations into high-quality research tasks, ensuring the traceability of
research task formulation while filtering noise. With the MAHTG system, we
curate DeepResearch Arena with over 10,000 high-quality research tasks from
over 200 academic seminars, spanning 12 disciplines, such as literature,
history, and science. Our extensive evaluation shows that DeepResearch Arena
presents substantial challenges for current state-of-the-art agents, with clear
performance gaps observed across different models.
DeepResearch Arena, a benchmark using academic seminar transcripts, provides high-quality research tasks to evaluate deep research agents across multiple disciplines.
DeepResearch Arena Multi-Agent Hierarchical Task Generation MAHTG research tasks academic seminars research agents