FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial
  Search and Reasoning
Search has emerged as core infrastructure for LLM-based agents and is widely
viewed as critical on the path toward more general intelligence. Finance is a
particularly demanding proving ground: analysts routinely conduct complex,
multi-step searches over time-sensitive, domain-specific data, making it ideal
for assessing both search proficiency and knowledge-grounded reasoning. Yet no
existing open financial datasets evaluate data searching capability of
end-to-end agents, largely because constructing realistic, complicated tasks
requires deep financial expertise and time-sensitive data is hard to evaluate.
We present FinSearchComp, the first fully open-source agent benchmark for
realistic, open-domain financial search and reasoning. FinSearchComp comprises
three tasks -- Time-Sensitive Data Fetching, Simple Historical Lookup, and
Complex Historical Investigation -- closely reproduce real-world financial
analyst workflows. To ensure difficulty and reliability, we engage 70
professional financial experts for annotation and implement a rigorous
multi-stage quality-assurance pipeline. The benchmark includes 635 questions
spanning global and Greater China markets, and we evaluate 21 models (products)
on it. Grok 4 (web) tops the global subset, approaching expert-level accuracy.
DouBao (web) leads on the Greater China subset. Experimental analyses show that
equipping agents with web search and financial plugins substantially improves
results on FinSearchComp, and the country origin of models and tools impact
performance significantly.By aligning with realistic analyst tasks and
providing end-to-end evaluation, FinSearchComp offers a professional,
high-difficulty testbed for complex financial search and reasoning.
68ccb6e23df9ac65e93dc60d
Liang Hu
68ccb6e23df9ac65e93dc60e
Jianpeng Jiao
68ccb6e23df9ac65e93dc60f
Jiashuo Liu
68ccb6e23df9ac65e93dc610
Yanle Ren
68ccb6e23df9ac65e93dc611
Zhoufutu Wen
68ccb6e23df9ac65e93dc612
Kaiyuan Zhang
68ccb6e23df9ac65e93dc613
Xuanliang Zhang
68ccb6e23df9ac65e93dc614
Xiang Gao
claimed_verified
2025-09-22T02:37:12.715Z
671068b8243baf4c58fc29c2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/leMkq-kw1icPo2bCLivGv.png
Xiang Gao
user
coffiney
68ccb6e23df9ac65e93dc615
Tianci He
68ccb6e23df9ac65e93dc616
Fei Hu
68ccb6e23df9ac65e93dc617
Yali Liao
68ccb6e23df9ac65e93dc618
Zaiyuan Wang
68ccb6e23df9ac65e93dc619
Chenghao Yang
68ccb6e23df9ac65e93dc61a
Qianyu Yang
68ccb6e23df9ac65e93dc61b
Mingren Yin
68ccb6e23df9ac65e93dc61c
Zhiyuan Zeng
68ccb6e23df9ac65e93dc61d
Ge Zhang
admin_assigned
2025-09-19T10:45:59.923Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
68ccb6e23df9ac65e93dc61e
Xinyi Zhang
68ccb6e23df9ac65e93dc61f
Xiying Zhao
68ccb6e23df9ac65e93dc620
Zhenwei Zhu
68ccb6e23df9ac65e93dc621
Hongseok Namkoong
68ccb6e23df9ac65e93dc622
Wenhao Huang
admin_assigned
2025-09-19T10:46:13.644Z
686c983f8c812deb40e83270
/avatars/7d1e4a38929b7ac79004463fd0060e9b.svg
Wenhao Huang
user
WenhaoHuang
68ccb6e23df9ac65e93dc623
Yuwen Tang
FinSearchComp: Towards a Realistic, Expert-Level Evaluation of Financial
  Search and Reasoning
Search has emerged as core infrastructure for LLM-based agents and is widely
viewed as critical on the path toward more general intelligence. Finance is a
particularly demanding proving ground: analysts routinely conduct complex,
multi-step searches over time-sensitive, domain-specific data, making it ideal
for assessing both search proficiency and knowledge-grounded reasoning. Yet no
existing open financial datasets evaluate data searching capability of
end-to-end agents, largely because constructing realistic, complicated tasks
requires deep financial expertise and time-sensitive data is hard to evaluate.
We present FinSearchComp, the first fully open-source agent benchmark for
realistic, open-domain financial search and reasoning. FinSearchComp comprises
three tasks -- Time-Sensitive Data Fetching, Simple Historical Lookup, and
Complex Historical Investigation -- closely reproduce real-world financial
analyst workflows. To ensure difficulty and reliability, we engage 70
professional financial experts for annotation and implement a rigorous
multi-stage quality-assurance pipeline. The benchmark includes 635 questions
spanning global and Greater China markets, and we evaluate 21 models (products)
on it. Grok 4 (web) tops the global subset, approaching expert-level accuracy.
DouBao (web) leads on the Greater China subset. Experimental analyses show that
equipping agents with web search and financial plugins substantially improves
results on FinSearchComp, and the country origin of models and tools impact
performance significantly.By aligning with realistic analyst tasks and
providing end-to-end evaluation, FinSearchComp offers a professional,
high-difficulty testbed for complex financial search and reasoning.
FinSearchComp is an open-source benchmark for evaluating financial search and reasoning capabilities of end-to-end agents, featuring realistic tasks and professional annotations.
LLM-based agents search proficiency knowledge-grounded reasoning FinSearchComp Time-Sensitive Data Fetching Simple Historical Lookup Complex Historical Investigation web search financial plugins