Towards Modular LLMs by Building and Reusing a Library of LoRAs
The growing number of parameter-efficient adaptations of a base large
language model (LLM) calls for studying whether we can reuse such trained
adapters to improve performance for new tasks. We study how to best build a
library of adapters given multi-task data and devise techniques for both
zero-shot and supervised task generalization through routing in such library.
We benchmark existing approaches to build this library and introduce
model-based clustering, MBC, a method that groups tasks based on the similarity
of their adapter parameters, indirectly optimizing for transfer across the
multi-task dataset. To re-use the library, we present a novel zero-shot routing
mechanism, Arrow, which enables dynamic selection of the most relevant adapters
for new inputs without the need for retraining. We experiment with several
LLMs, such as Phi-2 and Mistral, on a wide array of held-out tasks, verifying
that MBC-based adapters and Arrow routing lead to superior generalization to
new tasks. We make steps towards creating modular, adaptable LLMs that can
match or outperform traditional joint training.
664c0bf1cc9c64354ac9478e
Oleksiy Ostapenko
admin_assigned
2024-05-21T09:03:15.164Z
63be75e724e75d1efdf6ae36
/avatars/62d031d7fdb0d0747476d52c78dcdb18.svg
Oleksiy Ostapenko
user
ostapeno
664c0bf1cc9c64354ac9478f
Zhan Su
admin_assigned
2024-05-21T09:03:36.301Z
64dbf68ce1e953158d591adb
/avatars/30f48e9baaf2fbc7a9b0412a99b50bf2.svg
Zhan Su
user
zhan1993
664c0bf1cc9c64354ac94790
Edoardo Maria Ponti
admin_assigned
2024-05-21T09:03:42.851Z
60809ad44ad99100d63ce36a
https://cdn-avatars.huggingface.co/v1/production/uploads/1619040921084-noauth.jpeg
Edoardo Maria Ponti
user
ducdauge
664c0bf1cc9c64354ac94791
Laurent Charlin
664c0bf1cc9c64354ac94792
Nicolas Le Roux
admin_assigned
2024-05-21T09:03:57.607Z
6380eaa5b2552a70c4a19205
/avatars/99f601bf922996a51fe9782cd20afd28.svg
Nicolas Le Roux
user
nicolouchka
664c0bf1cc9c64354ac94793
Matheus Pereira
664c0bf1cc9c64354ac94794
Lucas Caccia
admin_assigned
2024-05-21T09:04:31.941Z
63cde3b2ff7cd335f0ea4441
/avatars/7a067499b7141efb4f2e7eb97d3807d5.svg
Lucas Caccia
user
pclucas14
664c0bf1cc9c64354ac94795
Alessandro Sordoni
extracted_confirmed
2024-05-21T02:53:08.598Z
637c27e5e74a3e06aacfd7b4
/avatars/27dcca787b11d880c0ad851e83a84e87.svg
Alessandro Sordoni
user
sordonia
Towards Modular LLMs by Building and Reusing a Library of LoRAs
The growing number of parameter-efficient adaptations of a base large
language model (LLM) calls for studying whether we can reuse such trained
adapters to improve performance for new tasks. We study how to best build a
library of adapters given multi-task data and devise techniques for both
zero-shot and supervised task generalization through routing in such library.
We benchmark existing approaches to build this library and introduce
model-based clustering, MBC, a method that groups tasks based on the similarity
of their adapter parameters, indirectly optimizing for transfer across the
multi-task dataset. To re-use the library, we present a novel zero-shot routing
mechanism, Arrow, which enables dynamic selection of the most relevant adapters
for new inputs without the need for retraining. We experiment with several
LLMs, such as Phi-2 and Mistral, on a wide array of held-out tasks, verifying
that MBC-based adapters and Arrow routing lead to superior generalization to
new tasks. We make steps towards creating modular, adaptable LLMs that can
match or outperform traditional joint training.
A novel approach using model-based clustering and zero-shot routing improves the generalization of adapters in large language models for new tasks.
parameter-efficient adaptations large language model (LLM) adapters multi-task data zero-shot task generalization supervised task generalization model-based clustering MBC task similarity transfer learning zero-shot routing Arrow dynamic selection joint training