FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction
Future prediction is a complex task for LLM agents, requiring a high level of
analytical thinking, information gathering, contextual understanding, and
decision-making under uncertainty. Agents must not only gather and interpret
vast amounts of dynamic information but also integrate diverse data sources,
weigh uncertainties, and adapt predictions based on emerging trends, just as
human experts do in fields like politics, economics, and finance. Despite its
importance, no large-scale benchmark exists for evaluating agents on future
prediction, largely due to challenges in handling real-time updates and
retrieving timely, accurate answers. To address this, we introduce
FutureX, a dynamic and live evaluation benchmark specifically
designed for LLM agents performing future prediction tasks. FutureX is the
largest and most diverse live benchmark for future prediction, supporting
real-time daily updates and eliminating data contamination through an automated
pipeline for question gathering and answer collection. We evaluate 25 LLM/agent
models, including those with reasoning, search capabilities, and integration of
external tools such as the open-source Deep Research Agent and closed-source
Deep Research models. This comprehensive evaluation assesses agents' adaptive
reasoning and performance in dynamic environments. Additionally, we provide
in-depth analyses of agents' failure modes and performance pitfalls in
future-oriented tasks, including the vulnerability to fake web pages and the
temporal validity. Our goal is to establish a dynamic, contamination-free
evaluation standard that drives the development of LLM agents capable of
performing at the level of professional human analysts in complex reasoning and
predictive thinking.
68a43789b65388761d0745dc
Zhiyuan Zeng
68a43789b65388761d0745dd
Jiashuo Liu
admin_assigned
2025-08-21T09:54:09.753Z
6842e440d029f9bcf58077b4
/avatars/bbdc6c603c38849d762055ae10d41d03.svg
jiashuo liu
user
liujiashuo77
68a43789b65388761d0745de
Siyuan Chen
68a43789b65388761d0745df
Tianci He
admin_assigned
2025-08-21T09:53:54.777Z
68a6b767fddc160a09aa883d
/avatars/0d8e94727f0e2298174c724afccf8c01.svg
Tianci He
user
Tianci-He
68a43789b65388761d0745e0
Yali Liao
68a43789b65388761d0745e1
Jinpeng Wang
68a43789b65388761d0745e2
Zaiyuan Wang
68a43789b65388761d0745e3
Yang Yang
claimed_verified
2025-08-22T11:12:42.735Z
68a6d5625c4ace95f9c8521b
https://cdn-avatars.huggingface.co/v1/production/uploads/68a6d5625c4ace95f9c8521b/-pPdstgnJN_M1kxjMek6Q.jpeg
yangyang
user
yangyang0606
68a43789b65388761d0745e4
Lingyue Yin
admin_assigned
2025-08-21T09:54:47.013Z
68a6d4bf99ba7d66206e2ce5
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/f29nNljyQNHSkovNqXBDy.png
Yin
user
YinLingyue
68a43789b65388761d0745e5
Mingren Yin
68a43789b65388761d0745e6
Zhenwei Zhu
admin_assigned
2025-08-21T09:55:03.392Z
6178b6a62be97faa212726ba
/avatars/07e216aeb6e45f6f9779238339ba8eec.svg
ZHENWEI ZHU
user
Nuori
68a43789b65388761d0745e7
Tianle Cai
admin_assigned
2025-08-21T09:55:10.542Z
64459357c525660aa20be337
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/1fryBXRuSvAgMIDkpWBpI.jpeg
Tianle Cai
user
tianlecai
68a43789b65388761d0745e8
Zehui Chen
admin_assigned
2025-08-21T09:55:16.941Z
64892d31cbda0d1cdb956897
/avatars/3cdafe03a8295124636347d15a099aaf.svg
Zehui Chen
user
lovesnowbest
68a43789b65388761d0745e9
Jiecao Chen
admin_assigned
2025-08-21T09:55:23.724Z
66df70fe5a0c5910d663160d
/avatars/980ca32bd0049ef5bbf002e7dc9f911c.svg
jiecao.chen
user
xmerge123
68a43789b65388761d0745ea
Yantao Du
68a43789b65388761d0745eb
Xiang Gao
claimed_verified
2025-09-22T02:37:16.596Z
671068b8243baf4c58fc29c2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/leMkq-kw1icPo2bCLivGv.png
Xiang Gao
user
coffiney
68a43789b65388761d0745ec
Jiacheng Guo
68a43789b65388761d0745ed
Liang Hu
68a43789b65388761d0745ee
Jianpeng Jiao
68a43789b65388761d0745ef
Xiangsheng Li
admin_assigned
2025-08-21T09:55:53.652Z
61a883b0d4b1c264b4b668d9
/avatars/87d569aaf821fda56e32849b728f021c.svg
Xiangsheng Li
user
lixsh6
68a43789b65388761d0745f0
Jingkai Liu
68a43789b65388761d0745f1
Shuang Ni
68a43789b65388761d0745f2
Zhoufutu Wen
claimed_verified
2025-08-27T15:00:14.223Z
6761461f6b3a8119fec25257
/avatars/559321b3dcf8d39b5f6e17a158f33e8e.svg
Maxwell Wen
user
MaxwellWen
68a43789b65388761d0745f3
Ge Zhang
claimed_verified
2025-08-20T08:52:32.340Z
638efcf4c67af472d316d424
/avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
Ge Zhang
user
zhangysk
68a43789b65388761d0745f4
Kaiyuan Zhang
68a43789b65388761d0745f5
Xin Zhou
68a43789b65388761d0745f6
Jose Blanchet
68a43789b65388761d0745f7
Xipeng Qiu
68a43789b65388761d0745f8
Mengdi Wang
68a43789b65388761d0745f9
Wenhao Huang
FutureX: An Advanced Live Benchmark for LLM Agents in Future Prediction
Future prediction is a complex task for LLM agents, requiring a high level of
analytical thinking, information gathering, contextual understanding, and
decision-making under uncertainty. Agents must not only gather and interpret
vast amounts of dynamic information but also integrate diverse data sources,
weigh uncertainties, and adapt predictions based on emerging trends, just as
human experts do in fields like politics, economics, and finance. Despite its
importance, no large-scale benchmark exists for evaluating agents on future
prediction, largely due to challenges in handling real-time updates and
retrieving timely, accurate answers. To address this, we introduce
FutureX, a dynamic and live evaluation benchmark specifically
designed for LLM agents performing future prediction tasks. FutureX is the
largest and most diverse live benchmark for future prediction, supporting
real-time daily updates and eliminating data contamination through an automated
pipeline for question gathering and answer collection. We evaluate 25 LLM/agent
models, including those with reasoning, search capabilities, and integration of
external tools such as the open-source Deep Research Agent and closed-source
Deep Research models. This comprehensive evaluation assesses agents' adaptive
reasoning and performance in dynamic environments. Additionally, we provide
in-depth analyses of agents' failure modes and performance pitfalls in
future-oriented tasks, including the vulnerability to fake web pages and the
temporal validity. Our goal is to establish a dynamic, contamination-free
evaluation standard that drives the development of LLM agents capable of
performing at the level of professional human analysts in complex reasoning and
predictive thinking.
FutureX is a dynamic, live benchmark for evaluating LLM agents in future prediction tasks, addressing challenges in real-time updates and data contamination.
LLM agents future prediction adaptive reasoning real-time updates data contamination automated pipeline question gathering answer collection failure modes performance pitfalls fake web pages temporal validity