QBasicVSR: Temporal Awareness Adaptation Quantization for Video Super-Resolution
振伟 张, Fanhua Shang, Hongying Liu, Liang Wan, Wei Feng, Yanming Hui
While model quantization has become pivotal for deploying super-resolution (SR) networks on mobile devices, existing works focus on quantization methods for image super-resolution. Different from image super-resolution, the temporal error propagation, shared temporal parameterization, and temporal metric mismatch significantly degrade the performance of a video SR model. To address these problems, we propose the first quantization method, QBasicVSR, for video super-resolution. A temporal awareness adaptation post-training quantization (PTQ) framework for video super-resolution with the flow-gradient video bit adaptation and temporal shared layer bit adaptation is presented. Moreover, we put forward a novel fine-tuning method for VSR with the supervision of the full-precision model. Our method achieves extraordinary performance with state-of-the-art efficient VSR approaches, delivering up to $\times$200 faster processing speed while utilizing only 1/8 of the GPU resources. Additionally, extensive experiments demonstrate that the proposed method significantly outperforms existing PTQ algorithms on various datasets. For instance, it attains a 2.53 dB increase on the UDM10 benchmark when quantizing BasicVSR to 4-bit with 100 unlabeled video clips. The code and models will be released on GitHub.