SCI-Verifier: Scientific Verifier with Thinking
As large language models (LLMs) are increasingly applied to scientific
reasoning, the complexity of answer formats and the diversity of equivalent
expressions make answer verification a critical yet challenging task. Existing
verification studies in scientific domains suffer from two major limitations:
(a) the absence of systematic evaluation standards and insufficient
disciplinary coverage, which hinders their comprehensive assessment; and (b)
heavy reliance on cumbersome rule design or prompt engineering, which reduces
their effectiveness in complex reasoning scenarios or limits their
cross-disciplinary generalization. To address these challenges, we propose
solutions at both the data and model levels. On the data side, we construct
SCI-VerifyBench, a cross-disciplinary benchmark covering mathematics, physics,
biology, chemistry, and general scientific QA. The benchmark is built from real
LLM responses and enhanced with domain-specific equivalence transformations
that generate challenging and realistic data. Model-based and expert
annotations ensure both quality and diversity, enabling rigorous evaluation of
verification ability. On the model side, we emphasize the importance of
reasoning for verification and introduce SCI-Verifier, a unified
reasoning-augmented verifier for scientific domains. Through post-training,
SCI-Verifier demonstrates strong logical reasoning and equivalence judgment
capabilities while maintaining concise and stable outputs. Together,
SCI-VerifyBench and SCI-Verifier provide a principled framework for scientific
verification, offering both systematic evaluation and practical pathways to
enhance the reliability and applicability of LLMs in scientific domains.
68db4b9dd2bf1f4b15ec743b
Shenghe Zheng
claimed_verified
2025-09-30T06:29:37.813Z
66e1557c75d9226ba13b38d2
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/vAwxAOynnsF2iqYeO3UBl.png
shenghe zheng
user
desimfj
68db4b9dd2bf1f4b15ec743c
Chenyu Huang
68db4b9dd2bf1f4b15ec743d
Fangchen Yu
68db4b9dd2bf1f4b15ec743e
Junchi Yao
68db4b9dd2bf1f4b15ec743f
Jingqi Ye
68db4b9dd2bf1f4b15ec7440
Tao Chen
68db4b9dd2bf1f4b15ec7441
Yun Luo
68db4b9dd2bf1f4b15ec7442
Ning Ding
68db4b9dd2bf1f4b15ec7443
LEI BAI
68db4b9dd2bf1f4b15ec7444
Ganqu Cui
68db4b9dd2bf1f4b15ec7445
Peng Ye
SCI-Verifier: Scientific Verifier with Thinking
As large language models (LLMs) are increasingly applied to scientific
reasoning, the complexity of answer formats and the diversity of equivalent
expressions make answer verification a critical yet challenging task. Existing
verification studies in scientific domains suffer from two major limitations:
(a) the absence of systematic evaluation standards and insufficient
disciplinary coverage, which hinders their comprehensive assessment; and (b)
heavy reliance on cumbersome rule design or prompt engineering, which reduces
their effectiveness in complex reasoning scenarios or limits their
cross-disciplinary generalization. To address these challenges, we propose
solutions at both the data and model levels. On the data side, we construct
SCI-VerifyBench, a cross-disciplinary benchmark covering mathematics, physics,
biology, chemistry, and general scientific QA. The benchmark is built from real
LLM responses and enhanced with domain-specific equivalence transformations
that generate challenging and realistic data. Model-based and expert
annotations ensure both quality and diversity, enabling rigorous evaluation of
verification ability. On the model side, we emphasize the importance of
reasoning for verification and introduce SCI-Verifier, a unified
reasoning-augmented verifier for scientific domains. Through post-training,
SCI-Verifier demonstrates strong logical reasoning and equivalence judgment
capabilities while maintaining concise and stable outputs. Together,
SCI-VerifyBench and SCI-Verifier provide a principled framework for scientific
verification, offering both systematic evaluation and practical pathways to
enhance the reliability and applicability of LLMs in scientific domains.
A framework combining SCI-VerifyBench and SCI-Verifier addresses challenges in verifying LLM-generated scientific answers through cross-disciplinary benchmarks and reasoning-augmented verification.
large language models scientific reasoning answer verification SCI-VerifyBench SCI-Verifier domain-specific equivalence transformations logical reasoning equivalence judgment