LiveResearchBench: A Live Benchmark for User-Centric Deep Research in
  the Wild
Deep research -- producing comprehensive, citation-grounded reports by
searching and synthesizing information from hundreds of live web sources --
marks an important frontier for agentic systems. To rigorously evaluate this
ability, four principles are essential: tasks should be (1) user-centric,
reflecting realistic information needs, (2) dynamic, requiring up-to-date
information beyond parametric knowledge, (3) unambiguous, ensuring consistent
interpretation across users, and (4) multi-faceted and search-intensive,
requiring search over numerous web sources and in-depth analysis. Existing
benchmarks fall short of these principles, often focusing on narrow domains or
posing ambiguous questions that hinder fair comparison. Guided by these
principles, we introduce LiveResearchBench, a benchmark of 100 expert-curated
tasks spanning daily life, enterprise, and academia, each requiring extensive,
dynamic, real-time web search and synthesis. Built with over 1,500 hours of
human labor, LiveResearchBench provides a rigorous basis for systematic
evaluation. To evaluate citation-grounded long-form reports, we introduce
DeepEval, a comprehensive suite covering both content- and report-level
quality, including coverage, presentation, citation accuracy and association,
consistency and depth of analysis. DeepEval integrates four complementary
evaluation protocols, each designed to ensure stable assessment and high
agreement with human judgments. Using LiveResearchBench and DeepEval, we
conduct a comprehensive evaluation of 17 frontier deep research systems,
including single-agent web search, single-agent deep research, and multi-agent
systems. Our analysis reveals current strengths, recurring failure modes, and
key system components needed to advance reliable, insightful deep research.
68f19a176e0bef323a68fc00
Jiayu Wang
claimed_verified
2025-10-18T04:49:53.348Z
651651f5d93a51ceda3021c3
https://cdn-avatars.huggingface.co/v1/production/uploads/651651f5d93a51ceda3021c3/FE2uGpTKBRWMKTDBv1H-g.png
Jiayu (Mila) Wang
user
MilaWang
68f19a176e0bef323a68fc01
Yifei Ming
68f19a176e0bef323a68fc02
Riya Dulepet
68f19a176e0bef323a68fc03
Qinglin Chen
68f19a176e0bef323a68fc04
Austin Xu
68f19a176e0bef323a68fc05
Zixuan Ke
68f19a176e0bef323a68fc06
Frederic Sala
68f19a176e0bef323a68fc07
Aws Albarghouthi
68f19a176e0bef323a68fc08
Caiming Xiong
68f19a176e0bef323a68fc09
Shafiq Joty
LiveResearchBench: A Live Benchmark for User-Centric Deep Research in
  the Wild
Deep research -- producing comprehensive, citation-grounded reports by
searching and synthesizing information from hundreds of live web sources --
marks an important frontier for agentic systems. To rigorously evaluate this
ability, four principles are essential: tasks should be (1) user-centric,
reflecting realistic information needs, (2) dynamic, requiring up-to-date
information beyond parametric knowledge, (3) unambiguous, ensuring consistent
interpretation across users, and (4) multi-faceted and search-intensive,
requiring search over numerous web sources and in-depth analysis. Existing
benchmarks fall short of these principles, often focusing on narrow domains or
posing ambiguous questions that hinder fair comparison. Guided by these
principles, we introduce LiveResearchBench, a benchmark of 100 expert-curated
tasks spanning daily life, enterprise, and academia, each requiring extensive,
dynamic, real-time web search and synthesis. Built with over 1,500 hours of
human labor, LiveResearchBench provides a rigorous basis for systematic
evaluation. To evaluate citation-grounded long-form reports, we introduce
DeepEval, a comprehensive suite covering both content- and report-level
quality, including coverage, presentation, citation accuracy and association,
consistency and depth of analysis. DeepEval integrates four complementary
evaluation protocols, each designed to ensure stable assessment and high
agreement with human judgments. Using LiveResearchBench and DeepEval, we
conduct a comprehensive evaluation of 17 frontier deep research systems,
including single-agent web search, single-agent deep research, and multi-agent
systems. Our analysis reveals current strengths, recurring failure modes, and
key system components needed to advance reliable, insightful deep research.
LiveResearchBench and DeepEval provide a comprehensive framework for evaluating deep research systems across various domains, focusing on real-time web search, synthesis, and citation-grounded long-form reports.
LiveResearchBench DeepEval deep research systems single-agent web search single-agent deep research multi-agent systems citation-grounded long-form reports real-time web search synthesis content-level quality report-level quality coverage presentation citation accuracy association consistency depth of analysis