Shepherd: A Critic for Language Model Generation
As large language models improve, there is increasing interest in techniques
that leverage these models' capabilities to refine their own outputs. In this
work, we introduce Shepherd, a language model specifically tuned to critique
responses and suggest refinements, extending beyond the capabilities of an
untuned model to identify diverse errors and provide suggestions to remedy
them. At the core of our approach is a high quality feedback dataset, which we
curate from community feedback and human annotations. Even though Shepherd is
small (7B parameters), its critiques are either equivalent or preferred to
those from established models including ChatGPT. Using GPT-4 for evaluation,
Shepherd reaches an average win-rate of 53-87% compared to competitive
alternatives. In human evaluation, Shepherd strictly outperforms other models
and on average closely ties with ChatGPT.
64d4423823996fa86776ef1c
Tianlu Wang
admin_assigned
2023-08-10T12:56:16.108Z
63f6a0e6b4c9a104f4b95d86
/avatars/660c3e44bea976f3e0560bf95466b03c.svg
Tianlu Wang
user
Tianlu
64d4423823996fa86776ef1d
Ping Yu
claimed_verified
2023-08-11T02:11:32.520Z
646c004e31968a60a022590e
/avatars/349e2ed4feef1be050fd9903dccec1e1.svg
Ping Yu
user
pingyu
64d4423823996fa86776ef1e
Xiaoqing Ellen Tan
claimed_verified
2023-08-11T02:11:36.576Z
6345ab8ffe134dfd7a0d1059
/avatars/159a364ac4f8a2ed001346d4cbb7f2e3.svg
Ellen Tan
user
ellenxtan
64d4423823996fa86776ef1f
Sean O'Brien
admin_assigned
2023-08-10T13:34:00.483Z
63f5451571a5d395c7212c94
/avatars/05e199ab2b5e2fe5403fc63b8825daf7.svg
Sean O'Brien
user
Sean-OB
64d4423823996fa86776ef20
Ramakanth Pasunuru
admin_assigned
2023-08-10T13:34:17.793Z
63b4be2334df1f9a90a1983c
/avatars/9cdfa3492c87aabdacab684b80628f14.svg
Ramakanth Pasunuru
user
rpasunuru
64d4423823996fa86776ef21
Jane Dwivedi-Yu
64d4423823996fa86776ef22
Olga Golovneva
admin_assigned
2023-08-10T13:34:45.589Z
6318b1e06e6bd039440871d6
/avatars/2d9f96eb4092a9d1734dcc09303b259a.svg
Olga Golovneva
user
Golovneva
64d4423823996fa86776ef23
Luke Zettlemoyer
64d4423823996fa86776ef24
Maryam Fazel-Zarandi
64d4423823996fa86776ef25
Asli Celikyilmaz
Shepherd: A Critic for Language Model Generation
As large language models improve, there is increasing interest in techniques
that leverage these models' capabilities to refine their own outputs. In this
work, we introduce Shepherd, a language model specifically tuned to critique
responses and suggest refinements, extending beyond the capabilities of an
untuned model to identify diverse errors and provide suggestions to remedy
them. At the core of our approach is a high quality feedback dataset, which we
curate from community feedback and human annotations. Even though Shepherd is
small (7B parameters), its critiques are either equivalent or preferred to
those from established models including ChatGPT. Using GPT-4 for evaluation,
Shepherd reaches an average win-rate of 53-87% compared to competitive
alternatives. In human evaluation, Shepherd strictly outperforms other models
and on average closely ties with ChatGPT.
Shepherd, a small language model tuned for critique, outperforms or ties with larger models like ChatGPT in refining language model outputs using a high-quality feedback dataset.
language model critique feedback dataset GPT-4 human evaluation ChatGPT win-rate