Ensemble-Instruct: Generating Instruction-Tuning Data with a
  Heterogeneous Mixture of LMs
Using in-context learning (ICL) for data generation, techniques such as
Self-Instruct (Wang et al., 2023) or the follow-up Alpaca (Taori et al., 2023)
can train strong conversational agents with only a small amount of human
supervision. One limitation of these approaches is that they resort to very
large language models (around 175B parameters) that are also proprietary and
non-public. Here we explore the application of such techniques to language
models that are much smaller (around 10B--40B parameters) and have permissive
licenses. We find the Self-Instruct approach to be less effective at these
sizes and propose new ICL methods that draw on two main ideas: (a)
Categorization and simplification of the ICL templates to make prompt learning
easier for the LM, and (b) Ensembling over multiple LM outputs to help select
high-quality synthetic examples. Our algorithm leverages the 175 Self-Instruct
seed tasks and employs separate pipelines for instructions that require an
input and instructions that do not. Empirical investigations with different LMs
show that: (1) Our proposed method yields higher-quality instruction tuning
data than Self-Instruct, (2) It improves performances of both vanilla and
instruction-tuned LMs by significant margins, and (3) Smaller instruction-tuned
LMs generate more useful outputs than their larger un-tuned counterparts. Our
codebase is available at https://github.com/IBM/ensemble-instruct.
65374541fa9f02750d26d086
Young-Suk Lee
admin_assigned
2023-10-24T08:51:10.070Z
64ba9290cefbfd87d391b11d
/avatars/557e7b850c7cfacd81738bb33d7e9b16.svg
Young-Suk Lee
user
samiangel
65374541fa9f02750d26d087
Md Arafat Sultan
65374541fa9f02750d26d088
Yousef El-Kurdi
65374541fa9f02750d26d089
Tahira Naseem Asim Munawar
65374541fa9f02750d26d08a
Radu Florian
admin_assigned
2023-10-24T08:51:32.325Z
642ae74b25442667e9bd72ed
/avatars/6c0898227e49f69eded5c7d7ed0242b9.svg
Radu Florian
user
hansolosan
65374541fa9f02750d26d08b
Salim Roukos
admin_assigned
2023-10-24T08:51:39.449Z
634d5b77fe861cc73a319aa2
/avatars/963d60bd4346820547f0cf7f0da68457.svg
Salim Roukos
user
roukos
65374541fa9f02750d26d08c
Ramón Fernandez Astudillo
Ensemble-Instruct: Generating Instruction-Tuning Data with a
  Heterogeneous Mixture of LMs
Using in-context learning (ICL) for data generation, techniques such as
Self-Instruct (Wang et al., 2023) or the follow-up Alpaca (Taori et al., 2023)
can train strong conversational agents with only a small amount of human
supervision. One limitation of these approaches is that they resort to very
large language models (around 175B parameters) that are also proprietary and
non-public. Here we explore the application of such techniques to language
models that are much smaller (around 10B--40B parameters) and have permissive
licenses. We find the Self-Instruct approach to be less effective at these
sizes and propose new ICL methods that draw on two main ideas: (a)
Categorization and simplification of the ICL templates to make prompt learning
easier for the LM, and (b) Ensembling over multiple LM outputs to help select
high-quality synthetic examples. Our algorithm leverages the 175 Self-Instruct
seed tasks and employs separate pipelines for instructions that require an
input and instructions that do not. Empirical investigations with different LMs
show that: (1) Our proposed method yields higher-quality instruction tuning
data than Self-Instruct, (2) It improves performances of both vanilla and
instruction-tuned LMs by significant margins, and (3) Smaller instruction-tuned
LMs generate more useful outputs than their larger un-tuned counterparts. Our
codebase is available at https://github.com/IBM/ensemble-instruct.
A new in-context learning method is proposed to generate high-quality instruction tuning data for smaller, open-access language models, improving their performance.
in-context learning ICL Self-Instruct Alpaca large language models small language models prompt learning ensembling instruction tuning codebase