Becoming self-instruct: introducing early stopping criteria for minimal
  instruct tuning
In this paper, we introduce the Instruction Following Score (IFS), a metric
that detects language models' ability to follow instructions. The metric has a
dual purpose. First, IFS can be used to distinguish between base and instruct
models. We benchmark publicly available base and instruct models, and show that
the ratio of well formatted responses to partial and full sentences can be an
effective measure between those two model classes. Secondly, the metric can be
used as an early stopping criteria for instruct tuning. We compute IFS for
Supervised Fine-Tuning (SFT) of 7B and 13B LLaMA models, showing that models
learn to follow instructions relatively early in the training process, and the
further finetuning can result in changes in the underlying base model
semantics. As an example of semantics change we show the objectivity of model
predictions, as defined by an auxiliary metric ObjecQA. We show that in this
particular case, semantic changes are the steepest when the IFS tends to
plateau. We hope that decomposing instruct tuning into IFS and semantic factors
starts a new trend in better controllable instruct tuning and opens
possibilities for designing minimal instruct interfaces querying foundation
models.
64ab91993c974392fbbedec5
Waseem AlShikh
claimed_verified
2023-07-11T07:54:33.150Z
60cd486d723acf5eb46fe8d3
https://cdn-avatars.huggingface.co/v1/production/uploads/60cd486d723acf5eb46fe8d3/Z1bD1kjvZ0QAOjZna41Xr.jpeg
Waseem AlShikh
user
wassemgtk
64ab91993c974392fbbedec6
Manhal Daaboul
64ab91993c974392fbbedec7
Kirk Goddard
64ab91993c974392fbbedec8
Brock Imel
64ab91993c974392fbbedec9
Kiran Kamble
claimed_verified
2023-07-11T07:54:36.772Z
621d6f532165dc431641e438
/avatars/56ccef10a8426d7160ef3586a771bd63.svg
Kiran Kamble
user
kiranr
64ab91993c974392fbbedeca
Parikshith Kulkarni
claimed_verified
2023-07-13T08:33:02.718Z
627426cfc3b4e8bb86b4cf8f
/avatars/5cf841c7bb77bc83997061f1dadc4d87.svg
Parikshith Kulkarni
user
parikshithkulkarni
64ab91993c974392fbbedecb
Melisa Russak
admin_assigned
2024-08-28T07:46:49.629Z
60e61b3969bd0df25c9375da
https://cdn-avatars.huggingface.co/v1/production/uploads/1625692968400-noauth.jpeg
Melisa Russak
user
melisa
Becoming self-instruct: introducing early stopping criteria for minimal
  instruct tuning
In this paper, we introduce the Instruction Following Score (IFS), a metric
that detects language models' ability to follow instructions. The metric has a
dual purpose. First, IFS can be used to distinguish between base and instruct
models. We benchmark publicly available base and instruct models, and show that
the ratio of well formatted responses to partial and full sentences can be an
effective measure between those two model classes. Secondly, the metric can be
used as an early stopping criteria for instruct tuning. We compute IFS for
Supervised Fine-Tuning (SFT) of 7B and 13B LLaMA models, showing that models
learn to follow instructions relatively early in the training process, and the
further finetuning can result in changes in the underlying base model
semantics. As an example of semantics change we show the objectivity of model
predictions, as defined by an auxiliary metric ObjecQA. We show that in this
particular case, semantic changes are the steepest when the IFS tends to
plateau. We hope that decomposing instruct tuning into IFS and semantic factors
starts a new trend in better controllable instruct tuning and opens
possibilities for designing minimal instruct interfaces querying foundation
models.
The Instruction Following Score (IFS) is introduced to evaluate and control instruction-following ability in language models, aiding in distinguishing between base and instruct models as well as serving as an early stopping criterion in training.
Instruction Following Score (IFS) instruct models base models Supervised Fine-Tuning (SFT) LLaMA models semantic changes ObjecQA