StatEval: A Comprehensive Benchmark for Large Language Models in
  Statistics
Large language models (LLMs) have demonstrated remarkable advances in
mathematical and logical reasoning, yet statistics, as a distinct and
integrative discipline, remains underexplored in benchmarking efforts. To
address this gap, we introduce StatEval, the first comprehensive
benchmark dedicated to statistics, spanning both breadth and depth across
difficulty levels. StatEval consists of 13,817 foundational problems covering
undergraduate and graduate curricula, together with 2374 research-level proof
tasks extracted from leading journals. To construct the benchmark, we design a
scalable multi-agent pipeline with human-in-the-loop validation that automates
large-scale problem extraction, rewriting, and quality control, while ensuring
academic rigor. We further propose a robust evaluation framework tailored to
both computational and proof-based tasks, enabling fine-grained assessment of
reasoning ability. Experimental results reveal that while closed-source models
such as GPT5-mini achieve below 57\% on research-level problems, with
open-source models performing significantly lower. These findings highlight the
unique challenges of statistical reasoning and the limitations of current LLMs.
We expect StatEval to serve as a rigorous benchmark for advancing statistical
intelligence in large language models. All data and code are available on our
web platform: https://stateval.github.io/.
68ec7109cd07fb414898c9da
Yuchen Lu
68ec7109cd07fb414898c9db
Run Yang
68ec7109cd07fb414898c9dc
Yichen Zhang
68ec7109cd07fb414898c9dd
Shuguang Yu
68ec7109cd07fb414898c9de
Runpeng Dai
claimed_verified
2025-10-13T10:05:24.400Z
65037565da2d88e201f63b7a
/avatars/d1b6ce17236360e9583b8bb4cb87e506.svg
Runpeng Dai
user
Leo-Dai
68ec7109cd07fb414898c9df
Ziwei Wang
68ec7109cd07fb414898c9e0
Jiayi Xiang
68ec7109cd07fb414898c9e1
Wenxin E
68ec7109cd07fb414898c9e2
Siran Gao
68ec7109cd07fb414898c9e3
Xinyao Ruan
68ec7109cd07fb414898c9e4
Yirui Huang
68ec7109cd07fb414898c9e5
Chenjing Xi
68ec7109cd07fb414898c9e6
Haibo Hu
68ec7109cd07fb414898c9e7
Yueming Fu
68ec7109cd07fb414898c9e8
Qinglan Yu
68ec7109cd07fb414898c9e9
Xiaobing Wei
68ec7109cd07fb414898c9ea
Jiani Gu
68ec7109cd07fb414898c9eb
Rui Sun
68ec7109cd07fb414898c9ec
Jiaxuan Jia
68ec7109cd07fb414898c9ed
Fan Zhou
StatEval: A Comprehensive Benchmark for Large Language Models in
  Statistics
Large language models (LLMs) have demonstrated remarkable advances in
mathematical and logical reasoning, yet statistics, as a distinct and
integrative discipline, remains underexplored in benchmarking efforts. To
address this gap, we introduce StatEval, the first comprehensive
benchmark dedicated to statistics, spanning both breadth and depth across
difficulty levels. StatEval consists of 13,817 foundational problems covering
undergraduate and graduate curricula, together with 2374 research-level proof
tasks extracted from leading journals. To construct the benchmark, we design a
scalable multi-agent pipeline with human-in-the-loop validation that automates
large-scale problem extraction, rewriting, and quality control, while ensuring
academic rigor. We further propose a robust evaluation framework tailored to
both computational and proof-based tasks, enabling fine-grained assessment of
reasoning ability. Experimental results reveal that while closed-source models
such as GPT5-mini achieve below 57\% on research-level problems, with
open-source models performing significantly lower. These findings highlight the
unique challenges of statistical reasoning and the limitations of current LLMs.
We expect StatEval to serve as a rigorous benchmark for advancing statistical
intelligence in large language models. All data and code are available on our
web platform: https://stateval.github.io/.
StatEval is a comprehensive benchmark for statistical reasoning, covering foundational and research-level problems, and highlights the limitations of current LLMs in this domain.
large language models LLMs mathematical reasoning logical reasoning statistics benchmark foundational problems research-level proof tasks multi-agent pipeline human-in-the-loop validation evaluation framework computational tasks proof-based tasks reasoning ability GPT5-mini statistical intelligence