ACADREASON: Exploring the Limits of Reasoning Models with Academic
  Research Problems
In recent years, the research focus of large language models (LLMs) and
agents has shifted increasingly from demonstrating novel capabilities to
complex reasoning and tackling challenging tasks. However, existing evaluations
focus mainly on math/code contests or general tasks, while existing
multi-domain academic benchmarks lack sufficient reasoning depth, leaving the
field without a rigorous benchmark for high-level reasoning. To fill this gap,
we introduce the Acadreason benchmark, designed to evaluate the ability of LLMs
and agents to acquire and reason over academic knowledge. It consists of 50
expert-annotated academic problems across five high-reasoning domains,
including computer science, economics, law, mathematics, and philosophy. All
questions are sourced from top-tier publications in recent years and undergo
rigorous annotation and quality control to ensure they are both challenging and
answerable. We conduct systematic evaluations of over 10 mainstream LLMs and
agents. The results show that most LLMs scored below 20 points, with even the
cutting-edge GPT-5 achieving only 16 points. While agents achieved higher
scores, none exceeded 40 points. This demonstrates the current capability gap
between LLMs and agents in super-intelligent academic research tasks and
highlights the challenges of Acadreason.
68edd775de1fee572713a97f
Xin Gui
claimed_verified
2025-10-14T07:29:37.509Z
67dbf07f9d821d38905d145d
/avatars/e806467d2c0f4f642c8d4906b0855817.svg
guixin
user
Ross12
68edd775de1fee572713a980
King Zhu
claimed_verified
2025-10-14T07:29:32.982Z
6578265ddea7e2122d02f6ba
https://cdn-avatars.huggingface.co/v1/production/uploads/6578265ddea7e2122d02f6ba/Bh6JjoVF5ceLSjV7Z7nTk.jpeg
king zhu
user
kangz
68edd775de1fee572713a981
JinCheng Ren
claimed_verified
2025-10-14T07:29:35.249Z
6704ee27386892c420db1938
https://cdn-avatars.huggingface.co/v1/production/uploads/6704ee27386892c420db1938/lb5mtEwYhn47RawkynYPs.jpeg
JinCheng Ren
user
JinChengRen
68edd775de1fee572713a982
Qianben Chen
68edd775de1fee572713a983
Zekun Moore Wang
68edd775de1fee572713a984
Yizhi LI
68edd775de1fee572713a985
Xinpeng Liu
68edd775de1fee572713a986
Xiaowan Li
68edd775de1fee572713a987
Wenli Ren
68edd775de1fee572713a988
Linyu Miao
68edd775de1fee572713a989
Tianrui Qin
68edd775de1fee572713a98a
Ziqi Shu
68edd775de1fee572713a98b
He Zhu
68edd775de1fee572713a98c
Xiangru Tang
68edd775de1fee572713a98d
Dingfeng Shi
68edd775de1fee572713a98e
Jiaheng Liu
68edd775de1fee572713a98f
Yuchen Eleanor Jiang
68edd775de1fee572713a990
Minghao Liu
claimed_verified
2025-10-24T16:14:37.218Z
6417d9ea8f689506e7148417
https://cdn-avatars.huggingface.co/v1/production/uploads/6417d9ea8f689506e7148417/bAYcruWNw4WvmuQcGgcwC.jpeg
minghao
user
Liam-Liu
68edd775de1fee572713a991
Ge Zhang
68edd775de1fee572713a992
Wangchunshu Zhou
ACADREASON: Exploring the Limits of Reasoning Models with Academic
  Research Problems
In recent years, the research focus of large language models (LLMs) and
agents has shifted increasingly from demonstrating novel capabilities to
complex reasoning and tackling challenging tasks. However, existing evaluations
focus mainly on math/code contests or general tasks, while existing
multi-domain academic benchmarks lack sufficient reasoning depth, leaving the
field without a rigorous benchmark for high-level reasoning. To fill this gap,
we introduce the Acadreason benchmark, designed to evaluate the ability of LLMs
and agents to acquire and reason over academic knowledge. It consists of 50
expert-annotated academic problems across five high-reasoning domains,
including computer science, economics, law, mathematics, and philosophy. All
questions are sourced from top-tier publications in recent years and undergo
rigorous annotation and quality control to ensure they are both challenging and
answerable. We conduct systematic evaluations of over 10 mainstream LLMs and
agents. The results show that most LLMs scored below 20 points, with even the
cutting-edge GPT-5 achieving only 16 points. While agents achieved higher
scores, none exceeded 40 points. This demonstrates the current capability gap
between LLMs and agents in super-intelligent academic research tasks and
highlights the challenges of Acadreason.
The Acadreason benchmark evaluates LLMs and agents on high-level academic reasoning across multiple domains, revealing significant capability gaps.
large language models agents Acadreason benchmark academic knowledge computer science economics law mathematics philosophy top-tier publications systematic evaluations GPT-5