JudgeLRM: Large Reasoning Models as a Judge
The rise of Large Language Models (LLMs) as evaluators offers a scalable
alternative to human annotation, yet existing Supervised Fine-Tuning (SFT) for
judges approaches often fall short in domains requiring complex reasoning. In
this work, we investigate whether LLM judges truly benefit from enhanced
reasoning capabilities. Through a detailed analysis of reasoning requirements
across evaluation tasks, we reveal a negative correlation between SFT
performance gains and the proportion of reasoning-demanding samples -
highlighting the limitations of SFT in such scenarios. To address this, we
introduce JudgeLRM, a family of judgment-oriented LLMs trained using
reinforcement learning (RL) with judge-wise, outcome-driven rewards. JudgeLRM
models consistently outperform both SFT-tuned and state-of-the-art reasoning
models. Notably, JudgeLRM-3B surpasses GPT-4, and JudgeLRM-7B outperforms
DeepSeek-R1 by 2.79% in F1 score, particularly excelling in judge tasks
requiring deep reasoning.
67ec9bf3e58745dc7d652587
Nuo Chen
claimed_verified
2025-04-03T08:28:54.107Z
641ac2207c21ab946bf036e8
https://cdn-avatars.huggingface.co/v1/production/uploads/641ac2207c21ab946bf036e8/r6c9gpOrul0eC59d9e2Mo.png
Nuo Chen
user
nuojohnchen
67ec9bf3e58745dc7d652588
Zhiyuan Hu
admin_assigned
2025-04-02T09:44:43.089Z
64351475901c5734bcb64248
/avatars/12346d4301c1bfb00ce0ea128a93cc15.svg
Zhiyuan Hu
user
zhiyuanhucs
67ec9bf3e58745dc7d652589
Qingyun Zou
admin_assigned
2025-04-02T09:44:49.663Z
6730a1fed66bf1b6378cd451
/avatars/5ec9b7313213a951b7c325d35ca26692.svg
qy
user
qingyun777yes
67ec9bf3e58745dc7d65258a
Jiaying Wu
67ec9bf3e58745dc7d65258b
Qian Wang
67ec9bf3e58745dc7d65258c
Bryan Hooi
admin_assigned
2025-04-02T09:45:24.844Z
651d8032c50012d33e914f2f
/avatars/0a44c9f51fc50ce86582e328c361ea00.svg
Bryan Hooi
user
bhooi
67ec9bf3e58745dc7d65258d
Bingsheng He
JudgeLRM: Large Reasoning Models as a Judge
The rise of Large Language Models (LLMs) as evaluators offers a scalable
alternative to human annotation, yet existing Supervised Fine-Tuning (SFT) for
judges approaches often fall short in domains requiring complex reasoning. In
this work, we investigate whether LLM judges truly benefit from enhanced
reasoning capabilities. Through a detailed analysis of reasoning requirements
across evaluation tasks, we reveal a negative correlation between SFT
performance gains and the proportion of reasoning-demanding samples -
highlighting the limitations of SFT in such scenarios. To address this, we
introduce JudgeLRM, a family of judgment-oriented LLMs trained using
reinforcement learning (RL) with judge-wise, outcome-driven rewards. JudgeLRM
models consistently outperform both SFT-tuned and state-of-the-art reasoning
models. Notably, JudgeLRM-3B surpasses GPT-4, and JudgeLRM-7B outperforms
DeepSeek-R1 by 2.79% in F1 score, particularly excelling in judge tasks
requiring deep reasoning.
JudgeLRM models trained with reinforcement learning outperform existing models, especially in tasks requiring deep reasoning.
Large Language Models Supervised Fine-Tuning reinforcement learning judge-wise outcome-driven rewards F1 score DeepSeek-R1