Abstract
Equipment’s computing capability has been greatly enhanced at present, which helps deep learning achieve excellent results in various applications, such as super-resolution. However, for higher performance, lower model size and faster computing speed, model compression is widely applied to accomplish the goal. For instance, model quantization is a typical compression method, such as quantization aware training and etc. Quantization aware training can take more quantization loss due to data mapping in model training into account, clamping and approximating the data representation range when updating parameters, which introduces quantization errors into loss function. In the quantization process, we used a quantization strategy that we quantized the model in different stages of combination, and found that some stages of the two super-resolution models’ generators based on SRGAN and ESRGAN showed sensitivity to quantization during the process, which greatly reduced the performance. Therefore, according to the quantization sensitivity, we use higher bits integer quantization for the sensitive stage, and get the multi-precision quantized model. For quantizing the SR model automatically, we propose a multi-precision quantization framework in this paper according to the ratio of input and output channels in every stage in the model. We also have our work tested on eight classical data sets of super-resolution. Generally speaking, both the two models’ PI values approach the original model’s respectively.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fan, S., Fei, J., Shen, L.: Accelerating deep learning with a parallel mechanism using CPU + MIC. Int. J. Parallel Program. 46(4), 660–673 (2018)
Ledig, C., et al.: Photo-realistic single image super-resolution using a generative adversarial network. In: CVPR, pp. 105–114. IEEE Computer Society (2017)
Dong, C., Loy, C.C., He, C.C., Tang, X.: Image super-resolution using deep convolutional networks. CoRR, vol. abs/1501.00092 (2015)
Wang, X., et al.: ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11133, pp. 63–79. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11021-5_5
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9906, pp. 694–711. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46475-6_43
Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In: CVPR, pp. 2790–2798. IEEE Computer Society (2017)
Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: CVPR Workshops, pp. 1132–1140. IEEE Computer Society (2017)
Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard GAN. In: ICLR (Poster). OpenReview.net (2019)
Choi, J., Wang, Z., Venkataramani, S., Chuang, P.I., Srinivasan, V., Gopalakrishnan, K.: PACT: parameterized clipping activation for quantized neural networks. CoRR, vol. abs/1805.06085 (2018)
Courbariaux, M., Bengio, Y., David, J.: BinaryConnect: training deep neural networks with binary weights during propagations. In: NIPS, pp. 3123–3131 (2015)
Courbariaux, M., Bengio, Y.: BinaryNet: training deep neural networks with weights and activations constrained to +1 or \(-\)1. CoRR, vol. abs/1602.02830 (2016)
Wu, J., Leng, C., Wang, Y., Hu, Q., Cheng, J.: Quantized convolutional neural networks for mobile devices. In: CVPR, pp. 4820–4828. IEEE Computer Society (2016)
Rastegari, M., Ordonez, V., Redmon, J., Farhadi, A.: XNOR-Net: ImageNet classification using binary convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9908, pp. 525–542. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46493-0_32
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications. CoRR, vol. abs/1704.04861 (2017)
Sa, C.D., et al.: High-accuracy low-precision training. CoRR, vol. abs/1803.03383 (2018)
Chu, T., Luo, Q., Yang, J., Huang, X.: Mixed-precision quantized neural networks with progressively decreasing bitwidth. Pattern Recogn. 111, 107647 (2021)
Mishra, A.K., Nurvitadhi, E., Cook, J.J., Marr, D.: WRPN: wide reduced-precision networks. In: ICLR (Poster). OpenReview.net (2018)
Zhuang, B., Liu, L., Tan, M., Shen, C., Reid, I.D.: Training quantized neural networks with a full-precision auxiliary module. In: CVPR, pp. 1485–1494. Computer Vision Foundation/IEEE (2020)
Li, F., Liu, B.: Ternary weight networks. CoRR, vol. abs/1605.04711 (2016)
Zhou, A., Yao, A., Guo, Y., Xu, L., Chen, Y.: Incremental network quantization: towards lossless CNNs with low-precision weights. In: ICLR (Poster). OpenReview.net (2017)
Hubara, I., Courbariaux, M., Soudry, D., El-Yaniv, R., Bengio, Y.: Quantized neural networks: training neural networks with low precision weights and activations. CoRR, vol. abs/1609.07061 (2016)
Kim, N., Shin, D., Choi, W., Kim, G., Park, J.: Exploiting retraining-based mixed-precision quantization for low-cost DNN accelerator design. IEEE Trans. Neural Netw. Learn. Syst. 32(7), 2925–2938 (2021)
Li, M., Lin, J., Ding, Y., Liu, Z., Zhu, J., Han, S.: GAN compression: Efficient architectures for interactive conditional GANs. In: CVPR, pp. 5283–5293. Computer Vision Foundation/IEEE (2020)
Zhuang, B., Liu, J., Tan, M., Liu, L., Reid, I.D., Shen, C.: Effective training of convolutional neural networks with low-bitwidth weights and activations. CoRR, vol. abs/1908.04680 (2019)
Cai, H., Gan, C., Wang, T., Zhang, Z., Han, S.: Once-for-all: train one network and specialize it for efficient deployment. In: ICLR. OpenReview.net (2020)
Chang, S., et al.: MSP: an FPGA-specific mixed-scheme, multi-precision deep neural network quantization framework. CoRR, vol. abs/2009.07460 (2020)
Vasquez, K., Venkatesha, Y., Bhattacharjee, A., Moitra, A., Panda, P.: Activation density based mixed-precision quantization for energy efficient neural networks. CoRR, vol. abs/2101.04354 (2021)
Goodfellow, I.J., et al.: Generative adversarial nets. In: NIPS, pp. 2672–2680 (2014)
Lee, R., et al.: Journey towards tiny perceptual super-resolution. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12371, pp. 85–102. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58574-7_6
Ma, Y., Xiong, H., Hu, Z., Ma, L.: Efficient super resolution using binarized neural network. In: CVPR Workshops, pp. 694–703. Computer Vision Foundation/IEEE (2019)
Xin, J., Wang, N., Jiang, X., Li, J., Huang, H., Gao, X.: Binarized neural network for single image super resolution. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 91–107. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_6
Li, H., et al.: PAMS: quantized super-resolution via parameterized max scale. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12370, pp. 564–580. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58595-2_34
Jacob, B., et al.: Quantization and training of neural networks for efficient integer-arithmetic-only inference. CoRR, vol. abs/1712.05877 (2017)
Soudry, D., Hubara, I., Meir, R.: Expectation backpropagation: parameter-free training of multilayer neural networks with continuous or discrete weights. In: NIPS, pp. 963–971 (2014)
Yuan, N., Zhu, Z., Wu, X., Shen, L.: MMSR: a multi-model super resolution framework. In: Tang, X., Chen, Q., Bose, P., Zheng, W., Gaudiot, J.-L. (eds.) NPC 2019. LNCS, vol. 11783, pp. 197–208. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-30709-7_16
Yuan, N., Liu, J., Wang, Q., Shen, L.: Customizing super-resolution framework according to image features. In: ISPA/BDCloud/SocialCom/SustainCom, pp. 1189–1196. IEEE (2020)
Yuan, N., Zhang, D., Wang, Q., Shen, L.: A multi-model super-resolution training and reconstruction framework. In: He, X., Shao, E., Tan, G. (eds.) NPC 2020. LNCS, vol. 12639, pp. 105–116. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-79478-1_9
Imambi, S., Prakash, K.B., Kanagachidambaresan, G.R.: PyTorch. Programming with TensorFlow (2021)
Zhang, S., Qin, Z., Yang, Y., Shen, L., Wang, Z.: Transparent partial page migration between CPU and GPU. Front. Comput. Sci. 14(3), 1–13 (2019). https://doi.org/10.1007/s11704-018-7386-4
Acknowledgment
This work is supported by National Nature Science Foundation of China (Grant No. 62032001 and 61972407) and Key Laboratory Open Projects Grant No. SZU-GDPHPCL201903.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, J., Zhang, D., Wang, Q., Shen, L. (2022). A Multi-precision Quantized Super-Resolution Model Framework. In: Lai, Y., Wang, T., Jiang, M., Xu, G., Liang, W., Castiglione, A. (eds) Algorithms and Architectures for Parallel Processing. ICA3PP 2021. Lecture Notes in Computer Science(), vol 13155. Springer, Cham. https://doi.org/10.1007/978-3-030-95384-3_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-95384-3_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95383-6
Online ISBN: 978-3-030-95384-3
eBook Packages: Computer ScienceComputer Science (R0)