LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
Pretrained large language models (LLMs) are currently state-of-the-art for
solving the vast majority of natural language processing tasks. While many
real-world applications still require fine-tuning to reach satisfactory levels
of performance, many of them are in the low-data regime, making fine-tuning
challenging. To address this, we propose LLM2LLM, a targeted and iterative data
augmentation strategy that uses a teacher LLM to enhance a small seed dataset
by augmenting additional data that can be used for fine-tuning on a specific
task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data,
(2) evaluates and extracts data points that the model gets wrong, and (3) uses
a teacher LLM to generate synthetic data based on these incorrect data points,
which are then added back into the training data. This approach amplifies the
signal from incorrectly predicted data points by the LLM during training and
reintegrates them into the dataset to focus on more challenging examples for
the LLM. Our results show that LLM2LLM significantly enhances the performance
of LLMs in the low-data regime, outperforming both traditional fine-tuning and
other data augmentation baselines. LLM2LLM reduces the dependence on
labor-intensive data curation and paves the way for more scalable and
performant LLM solutions, allowing us to tackle data-constrained domains and
tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on
CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular
fine-tuning in the low-data regime using a LLaMA2-7B student model.
6600fcca4b3187d931c818cd
Nicholas Lee
admin_assigned
2024-03-25T09:50:14.495Z
656a4547996819a82836852d
/avatars/a5b522c18fced97140a156ff3e88d2e8.svg
Nicholas Lee
user
nicholaslee
6600fcca4b3187d931c818ce
Thanakul Wattanawong
admin_assigned
2024-03-25T09:50:29.744Z
6427e76f4fe87caede7e85df
/avatars/be55e57ae14ef817e4393bbfd2ff14d4.svg
Wattanawong
user
Thanakul
6600fcca4b3187d931c818cf
Sehoon Kim
claimed_verified
2024-03-26T07:54:28.631Z
6022be42aa1483851c6685eb
/avatars/42975163f9903dfc6ec48b596ca219c5.svg
Sehoon Kim
user
kssteven
6600fcca4b3187d931c818d0
Karttikeya Mangalam
6600fcca4b3187d931c818d1
Sheng Shen
6600fcca4b3187d931c818d2
Gopala Anumanchipali
admin_assigned
2024-03-25T10:01:05.102Z
60523e4aa7226b25aaeea2b8
/avatars/316ca348da91ebced86991f36150c959.svg
Gopala Anumanchipalli
user
gopalakr
6600fcca4b3187d931c818d3
Michael W. Mahoney
admin_assigned
2024-03-25T10:01:20.313Z
64e8b009bddc5b1072c428a5
/avatars/350bac7b36ba1db1ac024a0812aa27c4.svg
MICHAEL MAHONEY
user
Mahoney2022
6600fcca4b3187d931c818d4
Kurt Keutzer
admin_assigned
2024-03-25T10:01:29.251Z
6251bf4b183aa4266924ad91
https://cdn-avatars.huggingface.co/v1/production/uploads/1678041834400-6251bf4b183aa4266924ad91.jpeg
Kurt Keutzer
user
kurtkeutzer
6600fcca4b3187d931c818d5
Amir Gholami
admin_assigned
2024-03-25T10:01:38.368Z
64e456d0384b85306bf5510e
/avatars/6c838f810c5f655d6b0b0b40211cfad7.svg
Amir Gholami
user
amirgh
LLM2LLM: Boosting LLMs with Novel Iterative Data Enhancement
Pretrained large language models (LLMs) are currently state-of-the-art for
solving the vast majority of natural language processing tasks. While many
real-world applications still require fine-tuning to reach satisfactory levels
of performance, many of them are in the low-data regime, making fine-tuning
challenging. To address this, we propose LLM2LLM, a targeted and iterative data
augmentation strategy that uses a teacher LLM to enhance a small seed dataset
by augmenting additional data that can be used for fine-tuning on a specific
task. LLM2LLM (1) fine-tunes a baseline student LLM on the initial seed data,
(2) evaluates and extracts data points that the model gets wrong, and (3) uses
a teacher LLM to generate synthetic data based on these incorrect data points,
which are then added back into the training data. This approach amplifies the
signal from incorrectly predicted data points by the LLM during training and
reintegrates them into the dataset to focus on more challenging examples for
the LLM. Our results show that LLM2LLM significantly enhances the performance
of LLMs in the low-data regime, outperforming both traditional fine-tuning and
other data augmentation baselines. LLM2LLM reduces the dependence on
labor-intensive data curation and paves the way for more scalable and
performant LLM solutions, allowing us to tackle data-constrained domains and
tasks. We achieve improvements up to 24.2% on the GSM8K dataset, 32.6% on
CaseHOLD, 32.0% on SNIPS, 52.6% on TREC and 39.8% on SST-2 over regular
fine-tuning in the low-data regime using a LLaMA2-7B student model.
LLM2LLM enhances LLM performance in low-data scenarios by iteratively using a teacher model to augment the training dataset with synthetic data based on the student's prediction errors.
pretrained large language models LLMs fine-tuning low-data regime data augmentation seed dataset baseline student LLM teacher LLM synthetic data evaluation incorrect data points GSM8K CaseHOLD SNIPS TREC SST-2 LLaMA2-7B