On scalable oversight with weak LLMs judging strong LLMs
Scalable oversight protocols aim to enable humans to accurately supervise
superhuman AI. In this paper we study debate, where two AI's compete to
convince a judge; consultancy, where a single AI tries to convince a judge that
asks questions; and compare to a baseline of direct question-answering, where
the judge just answers outright without the AI. We use large language models
(LLMs) as both AI agents and as stand-ins for human judges, taking the judge
models to be weaker than agent models. We benchmark on a diverse range of
asymmetries between judges and agents, extending previous work on a single
extractive QA task with information asymmetry, to also include mathematics,
coding, logic and multimodal reasoning asymmetries. We find that debate
outperforms consultancy across all tasks when the consultant is randomly
assigned to argue for the correct/incorrect answer. Comparing debate to direct
question answering, the results depend on the type of task: in extractive QA
tasks with information asymmetry debate outperforms direct question answering,
but in other tasks without information asymmetry the results are mixed.
Previous work assigned debaters/consultants an answer to argue for. When we
allow them to instead choose which answer to argue for, we find judges are less
frequently convinced by the wrong answer in debate than in consultancy.
Further, we find that stronger debater models increase judge accuracy, though
more modestly than in previous studies.
668b5ea8c2759a7bde1ea83f
Zachary Kenton
668b5ea8c2759a7bde1ea840
Noah Y. Siegel
admin_assigned
2024-07-08T21:22:59.957Z
64996a8560738aeea781da8a
/avatars/902bf170665089fb425a9fe38bcb53d9.svg
Noah Siegel
user
noahysiegel
668b5ea8c2759a7bde1ea841
János Kramár
admin_assigned
2024-07-08T21:22:30.317Z
6647488b45ee22196e798ba6
/avatars/7c068c30dcd5f07e4d896fd1dce5be2c.svg
Janos Kramar
user
jkramar
668b5ea8c2759a7bde1ea842
Jonah Brown-Cohen
admin_assigned
2024-07-08T21:21:44.004Z
667e645f673ab9bcd53c5140
/avatars/7e32f67f47fc75aa8a33e662375ab464.svg
Jonah Brown-Cohen
user
jonahbc
668b5ea8c2759a7bde1ea843
Samuel Albanie
admin_assigned
2024-07-08T21:21:38.140Z
62f3efefd6ba2ee26651f44a
https://cdn-avatars.huggingface.co/v1/production/uploads/1660153837083-noauth.png
Samuel Albanie
user
albanie
668b5ea8c2759a7bde1ea844
Jannis Bulian
668b5ea8c2759a7bde1ea845
Rishabh Agarwal
admin_assigned
2024-07-08T21:21:11.541Z
65318c1022350ffeaf7726ef
/avatars/9359e63d38cb69d08da2de07ae933885.svg
Rishabh Agarwal
user
rishabhagarwal
668b5ea8c2759a7bde1ea846
David Lindner
admin_assigned
2024-07-08T21:20:57.375Z
64aef034710608ee1a02b1ac
/avatars/0f8449bf7ee2a908962502ec863155be.svg
David Lindner
user
1obextiopo
668b5ea8c2759a7bde1ea847
Yunhao Tang
admin_assigned
2024-07-08T21:20:51.470Z
667c6733ed85453a289a38b2
/avatars/bf75c134186a8e225500372cbc8c2fcc.svg
Yunhao Tang
user
robinphysics
668b5ea8c2759a7bde1ea848
Noah D. Goodman
claimed_verified
2024-11-15T09:46:25.192Z
67321274c1f20c742bcf7a8d
https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/ltcQhre6eDRVzn6Vbbyhu.png
Noah D. Goodman
user
ngoodman
668b5ea8c2759a7bde1ea849
Rohin Shah
On scalable oversight with weak LLMs judging strong LLMs
Scalable oversight protocols aim to enable humans to accurately supervise
superhuman AI. In this paper we study debate, where two AI's compete to
convince a judge; consultancy, where a single AI tries to convince a judge that
asks questions; and compare to a baseline of direct question-answering, where
the judge just answers outright without the AI. We use large language models
(LLMs) as both AI agents and as stand-ins for human judges, taking the judge
models to be weaker than agent models. We benchmark on a diverse range of
asymmetries between judges and agents, extending previous work on a single
extractive QA task with information asymmetry, to also include mathematics,
coding, logic and multimodal reasoning asymmetries. We find that debate
outperforms consultancy across all tasks when the consultant is randomly
assigned to argue for the correct/incorrect answer. Comparing debate to direct
question answering, the results depend on the type of task: in extractive QA
tasks with information asymmetry debate outperforms direct question answering,
but in other tasks without information asymmetry the results are mixed.
Previous work assigned debaters/consultants an answer to argue for. When we
allow them to instead choose which answer to argue for, we find judges are less
frequently convinced by the wrong answer in debate than in consultancy.
Further, we find that stronger debater models increase judge accuracy, though
more modestly than in previous studies.
Debate between AI agents outperforms consultancy in diverse tasks and improves judge accuracy compared to direct question answering when agents choose their answers.
large language models debate consultancy direct question answering judge models agent models extractive QA information asymmetry mathematics coding logic multimodal reasoning