Self-Alignment with Instruction Backtranslation
We present a scalable method to build a high quality instruction following
language model by automatically labelling human-written text with corresponding
instructions. Our approach, named instruction backtranslation, starts with a
language model finetuned on a small amount of seed data, and a given web
corpus. The seed model is used to construct training examples by generating
instruction prompts for web documents (self-augmentation), and then selecting
high quality examples from among these candidates (self-curation). This data is
then used to finetune a stronger model. Finetuning LLaMa on two iterations of
our approach yields a model that outperforms all other LLaMa-based models on
the Alpaca leaderboard not relying on distillation data, demonstrating highly
effective self-alignment.
64d9acf1089bc502cef3610f
Xian Li
64d9acf1089bc502cef36110
Ping Yu
admin_assigned
2023-08-15T00:48:33.632Z
646c004e31968a60a022590e
/avatars/349e2ed4feef1be050fd9903dccec1e1.svg
Ping Yu
user
pingyu
64d9acf1089bc502cef36111
Chunting Zhou
64d9acf1089bc502cef36112
Timo Schick
admin_assigned
2023-08-15T00:49:02.082Z
613710606ecbbda5c86755c8
https://cdn-avatars.huggingface.co/v1/production/uploads/1636366232124-613710606ecbbda5c86755c8.png
Timo Schick
user
timoschick
64d9acf1089bc502cef36113
Luke Zettlemoyer
64d9acf1089bc502cef36114
Omer Levy
admin_assigned
2023-08-15T00:49:29.778Z
610b71d7f5dd7bcefedaff55
/avatars/b489c021e906010e661313da0b4aafab.svg
Omer Levy
user
omerlevy
64d9acf1089bc502cef36115
Jason Weston
admin_assigned
2023-08-15T00:49:50.913Z
62f023a36a027498eaa2f9cc
/avatars/8ac1c5c74d0957e3c6cc94b3a7795c37.svg
Jason Weston
user
spermwhale
64d9acf1089bc502cef36116
Mike Lewis
admin_assigned
2023-08-15T00:50:09.776Z
5e67eda71009063689407484
/avatars/6d7688faba888d463584eaa02ad011e3.svg
Mike Lewis
user
mikelewis0
Self-Alignment with Instruction Backtranslation
We present a scalable method to build a high quality instruction following
language model by automatically labelling human-written text with corresponding
instructions. Our approach, named instruction backtranslation, starts with a
language model finetuned on a small amount of seed data, and a given web
corpus. The seed model is used to construct training examples by generating
instruction prompts for web documents (self-augmentation), and then selecting
high quality examples from among these candidates (self-curation). This data is
then used to finetune a stronger model. Finetuning LLaMa on two iterations of
our approach yields a model that outperforms all other LLaMa-based models on
the Alpaca leaderboard not relying on distillation data, demonstrating highly
effective self-alignment.
A scalable instruction-following language model is built using auto-labelling and iterative self-augmentation and self-curation, outperforming other LLaMa-based models on Alpaca.
instruction backtranslation language model finetuned seed data web corpus instruction prompts self-augmentation self-curation LLaMa Alpaca leaderboard self-alignment