Trans-LoRA: towards data-free Transferable Parameter
  Efficient Finetuning
Low-rank adapters (LoRA) and their variants are popular parameter-efficient
fine-tuning (PEFT) techniques that closely match full model fine-tune
performance while requiring only a small number of additional parameters. These
additional LoRA parameters are specific to the base model being adapted. When
the base model needs to be deprecated and replaced with a new one, all the
associated LoRA modules need to be re-trained. Such re-training requires access
to the data used to train the LoRA for the original base model. This is
especially problematic for commercial cloud applications where the LoRA modules
and the base models are hosted by service providers who may not be allowed to
host proprietary client task data. To address this challenge, we propose
Trans-LoRA -- a novel method for lossless, nearly data-free transfer
of LoRAs across base models. Our approach relies on synthetic data to transfer
LoRA modules. Using large language models, we design a synthetic data generator
to approximate the data-generating process of the observed task data
subset. Training on the resulting synthetic dataset transfers LoRA modules to
new models. We show the effectiveness of our approach using both LLama and
Gemma model families. Our approach achieves lossless (mostly improved) LoRA
transfer between models within and across different base model families, and
even between different PEFT methods, on a wide variety of tasks.
66554b81ac73f7759fa84cad
Runqian Wang
claimed_verified
2024-05-28T10:36:17.540Z
656fd5c7e542334e136d88f6
/avatars/56f960d6b0d64f1cbb7f9f5ba69995e1.svg
Ray Wang
user
raywang4
66554b81ac73f7759fa84cae
Soumya Ghosh
claimed_verified
2024-05-30T07:24:21.234Z
65d11bc5bf849f3b79571771
/avatars/af8d2c02e2ecc39a319a0741aa830422.svg
Soumya Ghosh
user
leproquark
66554b81ac73f7759fa84caf
David Cox
66554b81ac73f7759fa84cb0
Diego Antognini
66554b81ac73f7759fa84cb1
Aude Oliva
66554b81ac73f7759fa84cb2
Rogerio Feris
admin_assigned
2024-05-28T10:26:55.838Z
6426053ca5ec4a5cbc516567
/avatars/7b841e0c58aa746e662fc8c78e3c5537.svg
Rogerio Feris
user
rferis
66554b81ac73f7759fa84cb3
Leonid Karlinsky
admin_assigned
2024-05-28T10:27:01.773Z
62558f7cebfc059817b06690
/avatars/40fbfdbeb0b33e21a48478deb2a742d1.svg
Leonid Karlinsky
user
leokarlin
Trans-LoRA: towards data-free Transferable Parameter
  Efficient Finetuning
Low-rank adapters (LoRA) and their variants are popular parameter-efficient
fine-tuning (PEFT) techniques that closely match full model fine-tune
performance while requiring only a small number of additional parameters. These
additional LoRA parameters are specific to the base model being adapted. When
the base model needs to be deprecated and replaced with a new one, all the
associated LoRA modules need to be re-trained. Such re-training requires access
to the data used to train the LoRA for the original base model. This is
especially problematic for commercial cloud applications where the LoRA modules
and the base models are hosted by service providers who may not be allowed to
host proprietary client task data. To address this challenge, we propose
Trans-LoRA -- a novel method for lossless, nearly data-free transfer
of LoRAs across base models. Our approach relies on synthetic data to transfer
LoRA modules. Using large language models, we design a synthetic data generator
to approximate the data-generating process of the observed task data
subset. Training on the resulting synthetic dataset transfers LoRA modules to
new models. We show the effectiveness of our approach using both LLama and
Gemma model families. Our approach achieves lossless (mostly improved) LoRA
transfer between models within and across different base model families, and
even between different PEFT methods, on a wide variety of tasks.
Trans-LoRA facilitates data-free transfer of LoRA modules across base models using synthetic data generated by large language models, enabling lossless or improved performance across various tasks and model families.
Low-rank adapters LoRA parameter-efficient fine-tuning PEFT base models re-training synthetic data data-free transfer LLama Gemma