Exploring Federated Pruning for Large Language Models
LLM pruning has emerged as a promising technology for compressing LLMs,
enabling their deployment on resource-limited devices. However, current
methodologies typically require access to public calibration samples, which can
be challenging to obtain in privacy-sensitive domains. To address this issue,
we introduce FedPrLLM, a comprehensive federated pruning framework designed for
the privacy-preserving compression of LLMs. In FedPrLLM, each client only needs
to calculate a pruning mask matrix based on its local calibration data and
share it with the server to prune the global model. This approach allows for
collaborative pruning of the global model with the knowledge of each client
while maintaining local data privacy. Additionally, we conduct extensive
experiments to explore various possibilities within the FedPrLLM framework,
including different comparison groups, pruning strategies, and the decision to
scale weights. Our extensive evaluation reveals that one-shot pruning with
layer comparison and no weight scaling is the optimal choice within the
FedPrLLM framework. We hope our work will help guide future efforts in pruning
LLMs in privacy-sensitive fields. Our code is available at
https://github.com/Pengxin-Guo/FedPrLLM.
682d6b06ec3b65b35772c0af
Pengxin Guo
claimed_verified
2025-05-21T08:40:11.271Z
668f440894dfc0ed1a7006ed
/avatars/fa0d328300b03bcbbf9b3a7532f28458.svg
Pengxin Guo
user
gpx333
682d6b06ec3b65b35772c0b0
Yinong Wang
admin_assigned
2025-05-21T10:12:52.463Z
641b065a1911d3be6742ef04
/avatars/bceb4ad2a278b6e2a40af0b89c4fb48e.svg
Yinong Wang
user
jcccy
682d6b06ec3b65b35772c0b1
Wei Li
682d6b06ec3b65b35772c0b2
Mengting Liu
admin_assigned
2025-05-21T10:13:13.691Z
678f8d4b1a5e393c1c285165
/avatars/533eb74376915eb63c03dc9b4df421f6.svg
MENGTINGLIU
user
MENGTINGLIU
682d6b06ec3b65b35772c0b3
Ming Li
682d6b06ec3b65b35772c0b4
Jinkai Zheng
682d6b06ec3b65b35772c0b5
Liangqiong Qu
admin_assigned
2025-05-21T10:12:59.427Z
663058bc2653ec94f4a6235f
/avatars/f55b8c3c8100d6b6d65ba61abc4fb014.svg
Liangqiong Qu
user
Liangqiong-QU
Exploring Federated Pruning for Large Language Models
LLM pruning has emerged as a promising technology for compressing LLMs,
enabling their deployment on resource-limited devices. However, current
methodologies typically require access to public calibration samples, which can
be challenging to obtain in privacy-sensitive domains. To address this issue,
we introduce FedPrLLM, a comprehensive federated pruning framework designed for
the privacy-preserving compression of LLMs. In FedPrLLM, each client only needs
to calculate a pruning mask matrix based on its local calibration data and
share it with the server to prune the global model. This approach allows for
collaborative pruning of the global model with the knowledge of each client
while maintaining local data privacy. Additionally, we conduct extensive
experiments to explore various possibilities within the FedPrLLM framework,
including different comparison groups, pruning strategies, and the decision to
scale weights. Our extensive evaluation reveals that one-shot pruning with
layer comparison and no weight scaling is the optimal choice within the
FedPrLLM framework. We hope our work will help guide future efforts in pruning
LLMs in privacy-sensitive fields. Our code is available at
https://github.com/Pengxin-Guo/FedPrLLM.
FedPrLLM is a federated pruning framework that enables privacy-preserving compression of large language models (LLMs) using local client data.
FedPrLLM federated pruning privacy-preserving compression LLMs calibration data pruning mask matrix layer comparison weight scaling