Great Models Think Alike and this Undermines AI Oversight
As Language Model (LM) capabilities advance, evaluating and supervising them
at scale is getting harder for humans. There is hope that other language models
can automate both these tasks, which we refer to as "AI Oversight". We study
how model similarity affects both aspects of AI oversight by proposing a
probabilistic metric for LM similarity based on overlap in model mistakes.
Using this metric, we first show that LLM-as-a-judge scores favor models
similar to the judge, generalizing recent self-preference results. Then, we
study training on LM annotations, and find complementary knowledge between the
weak supervisor and strong student model plays a crucial role in gains from
"weak-to-strong generalization". As model capabilities increase, it becomes
harder to find their mistakes, and we might defer more to AI oversight.
However, we observe a concerning trend -- model mistakes are becoming more
similar with increasing capabilities, pointing to risks from correlated
failures. Our work underscores the importance of reporting and correcting for
model similarity, especially in the emerging paradigm of AI oversight.
67a5b9107897c8f5406155e0
Shashwat Goel
admin_assigned
2025-02-07T10:39:36.508Z
6506832221ac448013f94995
https://cdn-avatars.huggingface.co/v1/production/uploads/6506832221ac448013f94995/sVUI1JV4Dxan5l-MqNze4.jpeg
Shashwat Goel
user
shash42
67a5b9107897c8f5406155e1
Joschka Struber
admin_assigned
2025-02-07T10:39:43.250Z
6728c6113d35dd53cfe9f30c
/avatars/7f93b9d41446cce382f63c78ca5059a1.svg
Joschka Strüber
user
Klingspor
67a5b9107897c8f5406155e2
Ilze Amanda Auzina
admin_assigned
2025-02-07T10:40:04.242Z
671b49503fd1d03dc69194b0
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/tnkR0j1VaWClUcumXcgjQ.png
Ilze Amanda Auzina
user
iaa01
67a5b9107897c8f5406155e3
Karuna K Chandra
67a5b9107897c8f5406155e4
Ponnurangam Kumaraguru
claimed_verified
2025-02-08T13:56:22.133Z
67a6a4b7f379cef464950268
https://cdn-avatars.huggingface.co/v1/production/uploads/noauth/yaMsT2jYsj6mvvYU1gvO_.jpeg
ponnurangam kumaraguru
user
pk-profgiri
67a5b9107897c8f5406155e5
Douwe Kiela
admin_assigned
2025-02-07T10:40:22.153Z
61dc997715b47073db1620dc
https://cdn-avatars.huggingface.co/v1/production/uploads/1641847245435-61dc997715b47073db1620dc.jpeg
Douwe Kiela
user
douwekiela
67a5b9107897c8f5406155e6
Ameya Prabhu
admin_assigned
2025-02-07T10:40:34.763Z
6464a0d41683d3c81f51924a
https://cdn-avatars.huggingface.co/v1/production/uploads/6464a0d41683d3c81f51924a/s7yYVwfUB4WOhVFJS6A6T.jpeg
Ameya Prabhu
user
AmeyaPrabhu
67a5b9107897c8f5406155e7
Matthias Bethge
67a5b9107897c8f5406155e8
Jonas Geiping
admin_assigned
2025-02-07T10:40:49.233Z
63d86dbf3130cadcaf8bdd11
/avatars/29d79a0c6dcec01111ef192fecd0fa7a.svg
Jonas Geiping
user
JonasGeiping
Great Models Think Alike and this Undermines AI Oversight
As Language Model (LM) capabilities advance, evaluating and supervising them
at scale is getting harder for humans. There is hope that other language models
can automate both these tasks, which we refer to as "AI Oversight". We study
how model similarity affects both aspects of AI oversight by proposing a
probabilistic metric for LM similarity based on overlap in model mistakes.
Using this metric, we first show that LLM-as-a-judge scores favor models
similar to the judge, generalizing recent self-preference results. Then, we
study training on LM annotations, and find complementary knowledge between the
weak supervisor and strong student model plays a crucial role in gains from
"weak-to-strong generalization". As model capabilities increase, it becomes
harder to find their mistakes, and we might defer more to AI oversight.
However, we observe a concerning trend -- model mistakes are becoming more
similar with increasing capabilities, pointing to risks from correlated
failures. Our work underscores the importance of reporting and correcting for
model similarity, especially in the emerging paradigm of AI oversight.
Evaluating and supervising advanced language models is increasingly challenging, and using other language models for oversight can introduce risks due to correlated mistakes.
AI Oversight probabilistic metric model mistakes LLM-as-a-judge self-preference weak supervisor strong student model weak-to-strong generalization correlated failures