BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
Pretrained large language models (LLMs) exhibit exceptional general language
processing capabilities but come with significant demands on memory and
computational resources. As a powerful compression technology, binarization can
extremely reduce model weights to a mere 1 bit, lowering the expensive
computation and memory requirements. However, existing quantization techniques
fall short of maintaining LLM performance under ultra-low bit-widths. In
response to this challenge, we present BiLLM, a groundbreaking 1-bit
post-training quantization scheme tailored for pretrained LLMs. Based on the
weight distribution of LLMs, BiLLM first identifies and structurally selects
salient weights, and minimizes the compression loss through an effective binary
residual approximation strategy. Moreover, considering the bell-shaped
distribution of the non-salient weights, we propose an optimal splitting search
to group and binarize them accurately. BiLLM achieving for the first time
high-accuracy inference (e.g. 8.41 perplexity on LLaMA2-70B) with only 1.08-bit
weights across various LLMs families and evaluation metrics, outperforms SOTA
quantization methods of LLM by significant margins. Moreover, BiLLM enables the
binarization process of the LLM with 7 billion weights within 0.5 hours on a
single GPU, demonstrating satisfactory time efficiency.
65c43a4ff19b126a3cd92d10
Wei Huang
claimed_verified
2024-04-22T06:55:21.980Z
656db3f53dc1d277e5a64410
https://cdn-avatars.huggingface.co/v1/production/uploads/656db3f53dc1d277e5a64410/9kiY2K3MCRcBDk7MrkTBK.png
Wei Huang
user
AaronHuangWei
65c43a4ff19b126a3cd92d11
Yangdong Liu
65c43a4ff19b126a3cd92d12
Haotong Qin
extracted_confirmed
2025-01-03T00:46:32.543Z
65c49589c0b1921e19260a8d
/avatars/7ce9af8c627f2a0c3db6bde82290ee1f.svg
Haotong Qin
user
HaotongQin
65c43a4ff19b126a3cd92d13
Ying Li
65c43a4ff19b126a3cd92d14
Shiming Zhang
65c43a4ff19b126a3cd92d15
Xianglong Liu
65c43a4ff19b126a3cd92d16
Michele Magno
65c43a4ff19b126a3cd92d17
Xiaojuan Qi
claimed_verified
2025-07-15T19:15:42.987Z
6875266f9cd3191dfddc7071
/avatars/64c581910833b111e9a7bae5b8740229.svg
xiaojuan qi
user
xjqi
BiLLM: Pushing the Limit of Post-Training Quantization for LLMs
Pretrained large language models (LLMs) exhibit exceptional general language
processing capabilities but come with significant demands on memory and
computational resources. As a powerful compression technology, binarization can
extremely reduce model weights to a mere 1 bit, lowering the expensive
computation and memory requirements. However, existing quantization techniques
fall short of maintaining LLM performance under ultra-low bit-widths. In
response to this challenge, we present BiLLM, a groundbreaking 1-bit
post-training quantization scheme tailored for pretrained LLMs. Based on the
weight distribution of LLMs, BiLLM first identifies and structurally selects
salient weights, and minimizes the compression loss through an effective binary
residual approximation strategy. Moreover, considering the bell-shaped
distribution of the non-salient weights, we propose an optimal splitting search
to group and binarize them accurately. BiLLM achieving for the first time
high-accuracy inference (e.g. 8.41 perplexity on LLaMA2-70B) with only 1.08-bit
weights across various LLMs families and evaluation metrics, outperforms SOTA
quantization methods of LLM by significant margins. Moreover, BiLLM enables the
binarization process of the LLM with 7 billion weights within 0.5 hours on a
single GPU, demonstrating satisfactory time efficiency.
BiLLM, a 1-bit post-training quantization scheme for pretrained LLMs, achieves high-accuracy inference with reduced computational and memory requirements.
binarization post-training quantization salient weights binary residual approximation optimal splitting search bell-shaped distribution perplexity LLaMA2-70B SOTA