Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models
Paper
•
2504.04823
•
Published
•
31
Quantization
-
-
GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers
Paper • 2210.17323 • Published • 8 -
AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration
Paper • 2306.00978 • Published • 11 -
The case for 4-bit precision: k-bit Inference Scaling Laws
Paper • 2212.09720 • Published • 3 -
LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale
Paper • 2208.07339 • Published • 5