Towards Robust and Efficient Continual Language Learning
As the application space of language models continues to evolve, a natural
question to ask is how we can quickly adapt models to new tasks. We approach
this classic question from a continual learning perspective, in which we aim to
continue fine-tuning models trained on past tasks on new tasks, with the goal
of "transferring" relevant knowledge. However, this strategy also runs the risk
of doing more harm than good, i.e., negative transfer. In this paper, we
construct a new benchmark of task sequences that target different possible
transfer scenarios one might face, such as a sequence of tasks with high
potential of positive transfer, high potential for negative transfer, no
expected effect, or a mixture of each. An ideal learner should be able to
maximally exploit information from all tasks that have any potential for
positive transfer, while also avoiding the negative effects of any distracting
tasks that may confuse it. We then propose a simple, yet effective, learner
that satisfies many of our desiderata simply by leveraging a selective strategy
for initializing new models from past task checkpoints. Still, limitations
remain, and we hope this benchmark can help the community to further build and
analyze such learners.
64af5c882cda6a37a48d04bd
Adam Fisch
64af5c882cda6a37a48d04be
Amal Rannen-Triki
64af5c882cda6a37a48d04bf
Razvan Pascanu
64af5c882cda6a37a48d04c0
Jörg Bornschein
claimed_verified
2024-06-25T08:58:56.944Z
6679bdb03043e7ed261a7699
/avatars/482ce2c4b2daf194161a0210e66d96de.svg
Jörg Bornschein
user
jbornschein
64af5c882cda6a37a48d04c1
Angeliki Lazaridou
64af5c882cda6a37a48d04c2
Elena Gribovskaya
64af5c882cda6a37a48d04c3
Marc'Aurelio Ranzato
Towards Robust and Efficient Continual Language Learning
As the application space of language models continues to evolve, a natural
question to ask is how we can quickly adapt models to new tasks. We approach
this classic question from a continual learning perspective, in which we aim to
continue fine-tuning models trained on past tasks on new tasks, with the goal
of "transferring" relevant knowledge. However, this strategy also runs the risk
of doing more harm than good, i.e., negative transfer. In this paper, we
construct a new benchmark of task sequences that target different possible
transfer scenarios one might face, such as a sequence of tasks with high
potential of positive transfer, high potential for negative transfer, no
expected effect, or a mixture of each. An ideal learner should be able to
maximally exploit information from all tasks that have any potential for
positive transfer, while also avoiding the negative effects of any distracting
tasks that may confuse it. We then propose a simple, yet effective, learner
that satisfies many of our desiderata simply by leveraging a selective strategy
for initializing new models from past task checkpoints. Still, limitations
remain, and we hope this benchmark can help the community to further build and
analyze such learners.
A benchmark of task sequences for continual learning evaluates transfer scenarios, and a selective initialization strategy for new models is proposed to mitigate negative transfer.
continual learning fine-tuning positive transfer negative transfer task sequences selective strategy task checkpoints