Loquetier: A Virtualized Multi-LoRA Framework for Unified LLM Fine-tuning and Serving
Yuchen Zhang, Hanyue Du, Chun Cao, Jingwei Xu
Low-Rank Adaptation (LoRA) has emerged as a widely adopted parameter-efficient fine-tuning (PEFT) technique to enhance the performance of large language models (LLMs) for downstream tasks. While prior work has explored integration strategies for LLM training and serving, there remains a gap in unifying fine-tuning and inference processes for LoRA-based models. In this paper, we present Loquetier, a virtualized multi-LoRA framework that enables unified LoRA fine-tuning and serving. Specifically, Loquetier consists of two main components: (1) a Virtualized Module that isolates model modifications to support multiple PEFT methods within a shared base model architecture,and (2) an optimized computation flow and kernel design that merges fine-tuning and inference paths in forward propagation, enabling efficient batching and minimizing kernel invocation overhead. Extensive experiments across three task settings demonstrate that Loquetier consistently outperforms existing baselines in performance and flexibility, achieving 3.0$\times$ the performance of the state-of-the-art co-serving system for the inference-only tasks, and 46.4$\times$ higher SLO attainment than PEFT in the unified tasks. The implementation of Loquetier is available at https://github.com/s3co3wjy5tr2bdfj/Loquetier.