Unlocking Continual Learning Abilities in Language Models
Language models (LMs) exhibit impressive performance and generalization
capabilities. However, LMs struggle with the persistent challenge of
catastrophic forgetting, which undermines their long-term sustainability in
continual learning (CL). Existing approaches usually address the issue by
incorporating old task data or task-wise inductive bias into LMs. However, old
data and accurate task information are often unavailable or costly to collect,
hindering the availability of current CL approaches for LMs. To address this
limitation, we introduce MIGU (MagnItude-based
Gradient Updating for continual learning), a
rehearsal-free and task-label-free method that only updates the model
parameters with large magnitudes of output in LMs' linear layers. MIGU is based
on our observation that the L1-normalized magnitude distribution of the output
in LMs' linear layers is different when the LM models deal with different task
data. By imposing this simple constraint on the gradient update process, we can
leverage the inherent behaviors of LMs, thereby unlocking their innate CL
abilities. Our experiments demonstrate that MIGU is universally applicable to
all three LM architectures (T5, RoBERTa, and Llama2), delivering
state-of-the-art or on-par performance across continual finetuning and
continual pre-training settings on four CL benchmarks. For example, MIGU brings
a 15.2% average accuracy improvement over conventional parameter-efficient
finetuning baselines in a 15-task CL benchmark. MIGU can also seamlessly
integrate with all three existing CL types to further enhance performance. Code
is available at https://github.com/wenyudu/MIGU{this https URL}.
667b8e9481adc76dc729773d
Wenyu Du
claimed_verified
2024-06-26T08:03:22.268Z
624c3d2ca19f20b197761ba9
/avatars/7a64b81c29f4f6700fa18effc5616865.svg
Wenyu Du
user
wydu
667b8e9481adc76dc729773e
Shuang Cheng
claimed_verified
2024-08-15T10:12:44.896Z
63478bc1e5c0717e673a086d
/avatars/66513149f2e92a65bec54364c2d360c0.svg
Cheng Shuang
user
chengs18
667b8e9481adc76dc729773f
Tongxu Luo
claimed_verified
2024-06-26T10:18:26.315Z
6421b07e918f0fd889f0a682
/avatars/314b55c2428426c846d9449f98db4355.svg
Tongxu Luo
user
Zeno-Luo
667b8e9481adc76dc7297740
Zihan Qiu
admin_assigned
2024-06-26T12:29:42.663Z
647ccbd6e07cf9bb2d485244
/avatars/e8915abaff04f6762247e196b7cf84df.svg
Zihan Qiu
user
QwQZh
667b8e9481adc76dc7297741
Zeyu Huang
claimed_verified
2024-08-14T12:42:36.119Z
64a5520dea3c5861303064b0
/avatars/7e618bcfc4d8d3291cf9bc8f2c4c1b15.svg
Zeyu hUANG
user
zeroyhuang
667b8e9481adc76dc7297742
Ka Chun Cheung
667b8e9481adc76dc7297743
Reynold Cheng
667b8e9481adc76dc7297744
Jie Fu
claimed_verified
2024-06-26T08:03:19.335Z
641a6895fb5ffff5ac79d593
https://cdn-avatars.huggingface.co/v1/production/uploads/641a6895fb5ffff5ac79d593/vxvwsto3llOEWGqQKGMYx.jpeg
Jie Fu
user
bigaidream
Unlocking Continual Learning Abilities in Language Models
Language models (LMs) exhibit impressive performance and generalization
capabilities. However, LMs struggle with the persistent challenge of
catastrophic forgetting, which undermines their long-term sustainability in
continual learning (CL). Existing approaches usually address the issue by
incorporating old task data or task-wise inductive bias into LMs. However, old
data and accurate task information are often unavailable or costly to collect,
hindering the availability of current CL approaches for LMs. To address this
limitation, we introduce MIGU (MagnItude-based
Gradient Updating for continual learning), a
rehearsal-free and task-label-free method that only updates the model
parameters with large magnitudes of output in LMs' linear layers. MIGU is based
on our observation that the L1-normalized magnitude distribution of the output
in LMs' linear layers is different when the LM models deal with different task
data. By imposing this simple constraint on the gradient update process, we can
leverage the inherent behaviors of LMs, thereby unlocking their innate CL
abilities. Our experiments demonstrate that MIGU is universally applicable to
all three LM architectures (T5, RoBERTa, and Llama2), delivering
state-of-the-art or on-par performance across continual finetuning and
continual pre-training settings on four CL benchmarks. For example, MIGU brings
a 15.2% average accuracy improvement over conventional parameter-efficient
finetuning baselines in a 15-task CL benchmark. MIGU can also seamlessly
integrate with all three existing CL types to further enhance performance. Code
is available at https://github.com/wenyudu/MIGU{this https URL}.
MIGU, a rehearsal-free and task-label-free method, enhances continual learning in language models by updating parameters with large magnitudes in linear layers, leading to state-of-the-art performance on multiple benchmarks.
language models catastrophic forgetting continual learning MIGU gradient updating L1-normalized magnitude distribution parameter-efficient fine-tuning T5 RoBERTa Llama2