Instruction Mining: High-Quality Instruction Data Selection for Large
  Language Models
Large language models typically undergo two training stages, pretraining and
finetuning. Despite that large-scale pretraining endows the model with strong
capabilities to generate natural language responses, these pretrained models
can still fail to understand human instructions at times. To enhance language
models' ability of interpreting and responding to instructions, instruction
finetuning has emerged as a critical method in this area. Recent studies found
that large language models can be finetuned to perform well even with a small
amount of high-quality instruction-following data. However, the selection of
high-quality datasets for finetuning language models still lacks clear
guidelines to follow. In this paper, we propose InstructMining, a linear rule
for evaluating instruction-following data quality. We formulate InstructMining
using specific natural language indicators. To investigate the relationship
between data quality and these indicators, we further conduct extensive
finetuning experiments. The experiment results are then applied to estimating
parameters in InstructMining. To further investigate its performance, we use
InstructMining to select high-quality data from unseen datasets. Results
demonstrate that InstructMining can help select relatively high-quality samples
from various instruction-following datasets. Compared to models finetuned on
unfiltered datasets, models finetuned on InstructMining selected datasets
perform better on 42.5% cases.
64af7e6a3b426e99c80e3be8
Yihan Cao
claimed_verified
2023-07-17T06:48:34.111Z
64b0332e252ed4adb7832978
https://cdn-avatars.huggingface.co/v1/production/uploads/64b0332e252ed4adb7832978/dNNWNNonYzes-u2BylPrJ.png
Yihan Cao
user
instructmining
64af7e6a3b426e99c80e3be9
Yanbin Kang
64af7e6a3b426e99c80e3bea
Lichao Sun
claimed_verified
2024-03-21T08:29:53.285Z
65a52766215aabac489e3468
/avatars/fe05e22cd7e12e961296426434e17c76.svg
Lichao Sun
user
sunlichao137
Instruction Mining: High-Quality Instruction Data Selection for Large
  Language Models
Large language models typically undergo two training stages, pretraining and
finetuning. Despite that large-scale pretraining endows the model with strong
capabilities to generate natural language responses, these pretrained models
can still fail to understand human instructions at times. To enhance language
models' ability of interpreting and responding to instructions, instruction
finetuning has emerged as a critical method in this area. Recent studies found
that large language models can be finetuned to perform well even with a small
amount of high-quality instruction-following data. However, the selection of
high-quality datasets for finetuning language models still lacks clear
guidelines to follow. In this paper, we propose InstructMining, a linear rule
for evaluating instruction-following data quality. We formulate InstructMining
using specific natural language indicators. To investigate the relationship
between data quality and these indicators, we further conduct extensive
finetuning experiments. The experiment results are then applied to estimating
parameters in InstructMining. To further investigate its performance, we use
InstructMining to select high-quality data from unseen datasets. Results
demonstrate that InstructMining can help select relatively high-quality samples
from various instruction-following datasets. Compared to models finetuned on
unfiltered datasets, models finetuned on InstructMining selected datasets
perform better on 42.5% cases.
InstructMining, a linear rule using natural language indicators, effectively selects high-quality instruction-following data for enhancing language model finetuning performance.
large language models pretraining finetuning instruction finetuning instruction-following data natural language indicators