Foundational Automatic Evaluators: Scaling Multi-Task Generative
  Evaluator Training for Reasoning-Centric Domains
Finetuning specialized generative evaluators has emerged as a popular
paradigm to meet the increasing demand for scalable evaluation during both
training and test-time. However, recent work has largely focused on applying
new methodology, such as reinforcement learning (RL), to training evaluators,
shying away from large-scale, data-driven development. In this work, we focus
on data scaling, curating a set of 2.5M samples spanning five unique evaluation
tasks (pairwise, step-level, reference-free and reference-based verification,
and single rating) and multiple domains focused on reasoning evaluation. With
our data, we train Foundational Automatic Reasoning Evaluators (FARE), a family
of 8B and 20B (with 3.6B active) parameter evaluators, with a simple iterative
rejection-sampling supervised finetuning (SFT) approach. FARE-8B challenges
larger specialized RL-trained evaluators and FARE-20B sets the new standard for
open-source evaluators, surpassing specialized 70B+ evaluators. Beyond static
benchmarks, we evaluate FARE in real-world tasks: As inference-time rerankers,
FARE-20B achieves near-oracle performance on MATH. As verifiers in RL training,
FARE improves the downstream RL-trained model performance by up to 14.1% vs.
string-matching verifiers. When initialized from FARE, a continually-finetuned
FARE-Code outperforms gpt-oss-20B by 65% on evaluating test-case quality.
68f6e6f524c44893631117e1
Austin Xu
68f6e6f524c44893631117e2
Xuan-Phi Nguyen
68f6e6f524c44893631117e3
Yilun Zhou
68f6e6f524c44893631117e4
Chien-Sheng Wu
68f6e6f524c44893631117e5
Caiming Xiong
68f6e6f524c44893631117e6
Shafiq Joty
Foundational Automatic Evaluators: Scaling Multi-Task Generative
  Evaluator Training for Reasoning-Centric Domains
Finetuning specialized generative evaluators has emerged as a popular
paradigm to meet the increasing demand for scalable evaluation during both
training and test-time. However, recent work has largely focused on applying
new methodology, such as reinforcement learning (RL), to training evaluators,
shying away from large-scale, data-driven development. In this work, we focus
on data scaling, curating a set of 2.5M samples spanning five unique evaluation
tasks (pairwise, step-level, reference-free and reference-based verification,
and single rating) and multiple domains focused on reasoning evaluation. With
our data, we train Foundational Automatic Reasoning Evaluators (FARE), a family
of 8B and 20B (with 3.6B active) parameter evaluators, with a simple iterative
rejection-sampling supervised finetuning (SFT) approach. FARE-8B challenges
larger specialized RL-trained evaluators and FARE-20B sets the new standard for
open-source evaluators, surpassing specialized 70B+ evaluators. Beyond static
benchmarks, we evaluate FARE in real-world tasks: As inference-time rerankers,
FARE-20B achieves near-oracle performance on MATH. As verifiers in RL training,
FARE improves the downstream RL-trained model performance by up to 14.1% vs.
string-matching verifiers. When initialized from FARE, a continually-finetuned
FARE-Code outperforms gpt-oss-20B by 65% on evaluating test-case quality.
FARE, a family of large-scale parameter evaluators, surpasses specialized RL-trained evaluators in both static benchmarks and real-world tasks through data-driven development and iterative rejection-sampling supervised finetuning.
generative evaluators reinforcement learning Foundational Automatic Reasoning Evaluators parameter evaluators iterative rejection-sampling supervised finetuning MATH continually-finetuned gpt-oss-20B