FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning
  Models on Automatically Verifiable Textual and Visual Questions
We conduct a moderate-scale contamination-free (to some extent) evaluation of
current large reasoning models (LRMs) with some preliminary findings. We also
release ROME, our evaluation benchmark for vision language models intended to
test reasoning from visual clues. We attach links to the benchmark, evaluation
data, and other updates on this website:
https://flageval-baai.github.io/LRM-Eval/
68d20da11ca7156988a8ed3c
Bowen Qin
68d20da11ca7156988a8ed3d
Chen Yue
68d20da11ca7156988a8ed3e
Fang Yin
68d20da11ca7156988a8ed3f
Hui Wang
68d20da11ca7156988a8ed40
JG Yao
68d20da11ca7156988a8ed41
Jiakang Liu
68d20da11ca7156988a8ed42
Jing-Shu Zheng
claimed_verified
2025-09-23T10:05:51.988Z
662f4fed259fa63f77da1f72
https://cdn-avatars.huggingface.co/v1/production/uploads/662f4fed259fa63f77da1f72/JOPCmhNeKE0d01tx-le5c.jpeg
jingshu
user
lilaczheng
68d20da11ca7156988a8ed43
Miguel Hu Chen
admin_assigned
2025-09-23T10:17:49.937Z
66f8b76d01bae9d4a4b8ce57
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/3fHkSeKq0YMPgkdYbrfZ7.png
Miguel Hu
user
miguelhuchen
68d20da11ca7156988a8ed44
Richeng Xuan
admin_assigned
2025-09-23T10:16:52.606Z
62fcd91b03f866462204b591
https://cdn-avatars.huggingface.co/v1/production/uploads/62fcd91b03f866462204b591/BkAVmJRKzBX_zRimf_yXY.png
Richeng Xuan
user
xuanricheng
68d20da11ca7156988a8ed45
Shibei Meng
68d20da11ca7156988a8ed46
Shiqi Zhou
admin_assigned
2025-09-23T10:17:33.950Z
6888907625619eb48fd2d72e
/avatars/554c601516adb9983ae03d3ebaa5d226.svg
Shiqi Zhou
user
stephaniezhou
68d20da11ca7156988a8ed47
Teng Dai
admin_assigned
2025-09-23T10:17:25.696Z
67411039d191d8c839ed65f4
/avatars/b91223e404e0568859fa6185b6cc6381.svg
Dai Teng
user
tengdai722
68d20da11ca7156988a8ed48
Tong-Shuai Ren
68d20da11ca7156988a8ed49
Wei Cui
68d20da11ca7156988a8ed4a
Xi Yang
claimed_verified
2025-09-25T08:02:08.369Z
637c248f9c470afa387fb699
/avatars/bb8db04ea1444eac0820fee3acd652c1.svg
Xi Yang
user
xiyang99
68d20da11ca7156988a8ed4b
Xialin Du
68d20da11ca7156988a8ed4c
Xiaojing Xu
68d20da11ca7156988a8ed4d
Xue Sun
68d20da11ca7156988a8ed4e
Xuejing Li
68d20da11ca7156988a8ed4f
Yaming Liu
admin_assigned
2025-09-23T10:16:44.686Z
681967d9704439a74a7e75c9
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/k-4GlEN6Tg1Mrvrl4JqBt.png
Yaming LIU
user
Yaming1
68d20da11ca7156988a8ed50
Yesheng Liu
68d20da11ca7156988a8ed51
Ying Liu
68d20da11ca7156988a8ed52
Yonghua Lin
admin_assigned
2025-09-23T10:15:04.277Z
629aa3155ab4232a3fe0893e
/avatars/cf2d4a9295b5da9e2e4d2278bbb36040.svg
Yonghua Lin
user
Yonghua
68d20da11ca7156988a8ed53
Yu Zhao
68d20da11ca7156988a8ed54
Yunduo Zhang
68d20da11ca7156988a8ed55
Yuwen Luo
68d20da11ca7156988a8ed56
Zheqi He
claimed_verified
2025-09-23T10:06:44.113Z
65b21047f5d76208991e463e
https://cdn-avatars.huggingface.co/v1/production/uploads/65b21047f5d76208991e463e/6ML4lLz-vUr1HdWR3Jo-L.jpeg
Zheqi He
user
philokey
68d20da11ca7156988a8ed57
Zhiyuan He
68d20da11ca7156988a8ed58
Zhongyuan Wang
FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning
  Models on Automatically Verifiable Textual and Visual Questions
We conduct a moderate-scale contamination-free (to some extent) evaluation of
current large reasoning models (LRMs) with some preliminary findings. We also
release ROME, our evaluation benchmark for vision language models intended to
test reasoning from visual clues. We attach links to the benchmark, evaluation
data, and other updates on this website:
https://flageval-baai.github.io/LRM-Eval/
A contamination-free evaluation of large reasoning models is conducted using the ROME benchmark, which tests reasoning from visual clues in vision language models.
large reasoning models ROME vision language models