R3: Robust Rubric-Agnostic Reward Models
Reward models are essential for aligning language model outputs with human
preferences, yet existing approaches often lack both controllability and
interpretability. These models are typically optimized for narrow objectives,
limiting their generalizability to broader downstream tasks. Moreover, their
scalar outputs are difficult to interpret without contextual reasoning. To
address these limitations, we introduce R3, a novel reward modeling framework
that is rubric-agnostic, generalizable across evaluation dimensions, and
provides interpretable, reasoned score assignments. R3 enables more transparent
and flexible evaluation of language models, supporting robust alignment with
diverse human values and use cases. Our models, data, and code are available as
open source at https://github.com/rubricreward/r3
682cb3213d53f7c2576ac370
David Anugraha
682cb3213d53f7c2576ac371
Zilu Tang
682cb3213d53f7c2576ac372
Lester James V. Miranda
claimed_verified
2025-05-28T09:01:27.643Z
634e20a0c1ce28f1de920cc4
https://cdn-avatars.huggingface.co/v1/production/uploads/1666064515342-noauth.jpeg
Lj V. Miranda
user
ljvmiranda921
682cb3213d53f7c2576ac373
Hanyang Zhao
682cb3213d53f7c2576ac374
Mohammad Rifqi Farhansyah
claimed_verified
2025-05-21T09:03:17.723Z
65af40481216d503271e7d95
https://cdn-avatars.huggingface.co/v1/production/uploads/65af40481216d503271e7d95/7BvAFiSHbL4rShMzdsLyd.png
Mohammad Rifqi Farhansyah
user
rifqifarhansyah
682cb3213d53f7c2576ac375
Garry Kuwanto
682cb3213d53f7c2576ac376
Derry Wijaya
682cb3213d53f7c2576ac377
Genta Indra Winata
R3: Robust Rubric-Agnostic Reward Models
Reward models are essential for aligning language model outputs with human
preferences, yet existing approaches often lack both controllability and
interpretability. These models are typically optimized for narrow objectives,
limiting their generalizability to broader downstream tasks. Moreover, their
scalar outputs are difficult to interpret without contextual reasoning. To
address these limitations, we introduce R3, a novel reward modeling framework
that is rubric-agnostic, generalizable across evaluation dimensions, and
provides interpretable, reasoned score assignments. R3 enables more transparent
and flexible evaluation of language models, supporting robust alignment with
diverse human values and use cases. Our models, data, and code are available as
open source at https://github.com/rubricreward/r3
R3 is a novel reward modeling framework that enhances controllability, interpretability, and flexibility in aligning language models with human preferences.
reward models alignment human preferences controllability interpretability generalizability rubric-agnostic evaluation dimensions reasoned score assignments transparent evaluation