Enabling Scalable Oversight via Self-Evolving Critic
Despite their remarkable performance, the development of Large Language
Models (LLMs) faces a critical challenge in scalable oversight: providing
effective feedback for tasks where human evaluation is difficult or where LLMs
outperform humans. While there is growing interest in using LLMs for critique,
current approaches still rely on human annotations or more powerful models,
leaving the issue of enhancing critique capabilities without external
supervision unresolved. We introduce SCRIT (Self-evolving CRITic), a framework
that enables genuine self-evolution of critique abilities. Technically, SCRIT
self-improves by training on synthetic data, generated by a contrastive-based
self-critic that uses reference solutions for step-by-step critique, and a
self-validation mechanism that ensures critique quality through correction
outcomes. Implemented with Qwen2.5-72B-Instruct, one of the most powerful LLMs,
SCRIT achieves up to a 10.3\% improvement on critique-correction and error
identification benchmarks. Our analysis reveals that SCRIT's performance scales
positively with data and model size, outperforms alternative approaches, and
benefits critically from its self-validation component.
67847bca43f26c67cc29d68f
Zhengyang Tang
admin_assigned
2025-01-13T09:20:56.766Z
64912976b95c3f0a1e6233cb
/avatars/c0615f8c6606073faffb419757d4e667.svg
Zhengyang Tang
user
tangzhy
67847bca43f26c67cc29d690
Ziniu Li
admin_assigned
2025-01-13T09:21:26.947Z
647c4c901f878439e2fd34d6
/avatars/b4399d210d7239d4662b11a4ee7b527d.svg
Ziniu Li
user
ziniuli
67847bca43f26c67cc29d691
Zhenyang Xiao
admin_assigned
2025-01-13T09:21:40.957Z
649d87b19f0ffe21ef26d36b
/avatars/2a0b47328a780f24a640933111fe163c.svg
xiaozhenyang
user
yeshoubaizi
67847bca43f26c67cc29d692
Tian Ding
67847bca43f26c67cc29d693
Ruoyu Sun
67847bca43f26c67cc29d694
Benyou Wang
admin_assigned
2025-01-13T09:22:43.093Z
637c6703ca8542a0ba900ccb
/avatars/288ed63a1efa566c3f01e850c6ba5dd5.svg
Wang
user
Benyou
67847bca43f26c67cc29d695
Dayiheng Liu
admin_assigned
2025-01-13T09:22:49.154Z
6434d4989bd5a84b5dd0b0f5
https://cdn-avatars.huggingface.co/v1/production/uploads/6434d4989bd5a84b5dd0b0f5/0Elf9qbfG9Hkgypm9pTGm.jpeg
Dayiheng Liu
user
Losin94
67847bca43f26c67cc29d696
Fei Huang
admin_assigned
2025-01-13T09:23:37.449Z
635b8b6a37c6a2c12e2cce00
/avatars/229fb72180529141515d1df797b33709.svg
Fei Huang
user
hzhwcmhf
67847bca43f26c67cc29d697
Tianyu Liu
67847bca43f26c67cc29d698
Bowen Yu
admin_assigned
2025-01-13T09:24:32.921Z
6438b43ab2ea24b52ebac2b9
/avatars/84133cd719a4b1e2f5c1a74178425f86.svg
Bowen Yu
user
bwy
67847bca43f26c67cc29d699
Junyang Lin
admin_assigned
2025-01-13T09:24:39.156Z
620760a26e3b7210c2ff1943
https://cdn-avatars.huggingface.co/v1/production/uploads/620760a26e3b7210c2ff1943/VC-rKqimF6yxGESNVlPoR.jpeg
Junyang Lin
user
JustinLin610
Enabling Scalable Oversight via Self-Evolving Critic
Despite their remarkable performance, the development of Large Language
Models (LLMs) faces a critical challenge in scalable oversight: providing
effective feedback for tasks where human evaluation is difficult or where LLMs
outperform humans. While there is growing interest in using LLMs for critique,
current approaches still rely on human annotations or more powerful models,
leaving the issue of enhancing critique capabilities without external
supervision unresolved. We introduce SCRIT (Self-evolving CRITic), a framework
that enables genuine self-evolution of critique abilities. Technically, SCRIT
self-improves by training on synthetic data, generated by a contrastive-based
self-critic that uses reference solutions for step-by-step critique, and a
self-validation mechanism that ensures critique quality through correction
outcomes. Implemented with Qwen2.5-72B-Instruct, one of the most powerful LLMs,
SCRIT achieves up to a 10.3\% improvement on critique-correction and error
identification benchmarks. Our analysis reveals that SCRIT's performance scales
positively with data and model size, outperforms alternative approaches, and
benefits critically from its self-validation component.
SCRIT, a self-evolving critique framework, enhances LLMs' critique capabilities using synthetic data and self-validation, achieving significant improvements in critique-correction and error identification benchmarks.
Large Language Models (LLMs) self-evolving synthetic data contrastive-based self-critic reference solutions step-by-step critique self-validation Qwen2.5-72B-Instruct critique-correction error identification benchmarks