EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction
Open-source foundation models have seen rapid adoption and development,
enabling powerful general-purpose capabilities across diverse domains. However,
fine-tuning large foundation models for domain-specific or personalized tasks
remains prohibitively expensive for most users due to the significant memory
overhead beyond that of inference. We introduce EMLoC, an Emulator-based
Memory-efficient fine-tuning framework with LoRA Correction, which enables
model fine-tuning within the same memory budget required for inference. EMLoC
constructs a task-specific light-weight emulator using activation-aware
singular value decomposition (SVD) on a small downstream calibration set.
Fine-tuning then is performed on this lightweight emulator via LoRA. To tackle
the misalignment between the original model and the compressed emulator, we
propose a novel compensation algorithm to correct the fine-tuned LoRA module,
which thus can be merged into the original model for inference. EMLoC supports
flexible compression ratios and standard training pipelines, making it
adaptable to a wide range of applications. Extensive experiments demonstrate
that EMLoC outperforms other baselines across multiple datasets and modalities.
Moreover, without quantization, EMLoC enables fine-tuning of a 38B model on a
single 24GB consumer GPU-bringing efficient and practical model adaptation to
individual users.
68529976a523e43c8c016672
Hsi-Che Lin
68529976a523e43c8c016673
Yu-Chu Yu
68529976a523e43c8c016674
Kai-Po Chang
68529976a523e43c8c016675
Yu-Chiang Frank Wang
EMLoC: Emulator-based Memory-efficient Fine-tuning with LoRA Correction
Open-source foundation models have seen rapid adoption and development,
enabling powerful general-purpose capabilities across diverse domains. However,
fine-tuning large foundation models for domain-specific or personalized tasks
remains prohibitively expensive for most users due to the significant memory
overhead beyond that of inference. We introduce EMLoC, an Emulator-based
Memory-efficient fine-tuning framework with LoRA Correction, which enables
model fine-tuning within the same memory budget required for inference. EMLoC
constructs a task-specific light-weight emulator using activation-aware
singular value decomposition (SVD) on a small downstream calibration set.
Fine-tuning then is performed on this lightweight emulator via LoRA. To tackle
the misalignment between the original model and the compressed emulator, we
propose a novel compensation algorithm to correct the fine-tuned LoRA module,
which thus can be merged into the original model for inference. EMLoC supports
flexible compression ratios and standard training pipelines, making it
adaptable to a wide range of applications. Extensive experiments demonstrate
that EMLoC outperforms other baselines across multiple datasets and modalities.
Moreover, without quantization, EMLoC enables fine-tuning of a 38B model on a
single 24GB consumer GPU-bringing efficient and practical model adaptation to
individual users.
EMLoC, an memory-efficient fine-tuning framework using activation-aware SVD and LoRA, allows model adaptation within inference memory constraints for diverse applications.
activation-aware singular value decomposition SVD Emulator-based Memory-efficient fine-tuning framework LoRA Correction LoRA fine-tuning model fine-tuning standard training pipelines model adaptation