Qiong Zhang
ABSTRACT
The performance of single image super-resolution (SISR) has improved significantly in recent years. However, most existing methods suffer from large model parameters, high computational costs, and slow inference times, making them unsuitable for resource-constrained devices. To optimize these limitations, we propose a new re-parameterizable residual multiple convolutions network (RepRMCN) that strikes a better balance between model performance, scale, and inference time. Our approach introduces a convolutional reparameterization strategy that combines multiple convolutionals into a single convolution after model training, improving model performance during training and reducing the parameters and inference time when deployed. We also designed a simple grouped structure with different convolutions that inspired by grouped convolution, Inception, and heterogeneous kernel-based convolution. Our proposed structure splits features into two groups and processes them using re-parameterizable multiple convolutions module and dilation convolution, respectively. Experimental results show that RepRMCN outperforms existing high-efficiency SR models in terms of parameter reduction and inference time while maintaining good performance. Specifically, when tested on the Urban100 dataset with a 4× scale factor, our RepRMCN has 495K parameters, which is 48K less than the current state-of-the-art model RLFN. The inference time on the GeForce GTX 1080Ti is 16.47ms, which is comparable to RLFN.
- Chao Dong, Chen Change Loy, Kaiming He and Xiaoou Tang. 2014. Learning a deep convolutional network for image super-resolution. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science (), vol 8692. Springer, Cham. https://doi.org/10.1007/978-3-319-10593-2_13Google Scholar
Cross Ref
- Chao Dong, Chen Change Loy, and Xiaoou Tang. 2016. Accelerating the super-resolution convolutional neural network. (eds) Computer Vision – ECCV 2016. ECCV 2016. Lecture Notes in Computer Science (), vol 9906. Springer, Cham. https://doi.org/10.1007/978-3-319-46475-6_25, 2016:391-407Google ScholarCross Ref
- Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Accurate image super-resolution using very deep convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, USA, 2016, pp. 1646-1654. https://doi.org/10.1109/CVPR.2016.181Google ScholarCross Ref
- Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Deeply-recursive convolutional network for image super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, USA, 2016, pp. 1637-1645. https://doi.org/10.1109/CVPR.2016.181Google ScholarCross Ref
- Wei-Sheng Lai, Jia-Bin Huang, Narendra Ahuja, and Ming-Hsuan Yang. 2017. Deep laplacian pyramid networks for fast and accurate super-resolution. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, USA, 2017, pp. 5835-5843. https://10.1109/CVPR.2017.618Google ScholarCross Ref
- Namhyuk Ahn, Byungkon Kang, and Kyung-Ah Sohn. 2018. Fast, accurate, and lightweight super-resolution with cascading residual network. (eds) Computer Vision – ECCV 2018. ECCV 2018. Lecture Notes in Computer Science (), vol 11214. Springer, Cham. https://doi.org/10.1007/978-3-030-01249-6_16Google ScholarDigital Library
- Zheng Hui, Xiumei Wang, and Xinbo Gao. 2018. Fast and accurate single image super-resolution via information distillation network. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, USA, 2018, pp. 723-731. https://doi.org/10.1109/CVPR.2018.00082Google ScholarCross Ref
- Zheng Hui, Xinbo Gao, Yunchu Yang, and Xiumei Wang. 2019. Lightweight Image Super-Resolution with Information Multi-distillation Network. In Proceedings of the 27th ACM International Conference on Multimedia (MM '19). Association for Computing Machinery, New York, NY, USA, 2024–2032. https://doi.org/10.1145/3343031.3351084Google ScholarDigital Library
- Jie Liu, Wenjie Zhang, Yuting Tang, Jie Tang, and Gangshan Wu. 2020. Residual feature aggregation network for image super-resolution. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. IEEE, Seattle, WA, USA, 2020, pp. 2356-2365. https://10.1109/CVPR42600.2020.00243Google ScholarCross Ref
- Hengyuan Zhao, Xiangtao Kong, Jingwen He, Yu Qiao, and Chao Dong. 2020. Efficient image super-resolution using pixel attention. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science (), vol 12537. Springer, Cham. https://doi.org/10.1007/978-3-030-67070-2_3Google ScholarDigital Library
- Jingyun Liang, Jiezhang Cao, Guolei Sun, Kai Zhang, Luc Van Gool, and Radu Timofte. 2021. Swinir: Image restoration using swin transformer. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, International Conference on Computer Vision Workshops (ICCVW), Montreal, BC, Canada, 2021, pp. 1833-1844. https://doi.org/10.1109/ICCVW54120.2021.00210Google ScholarCross Ref
- Fangyuan Kong, Mingxi Li, Songwei Liu, Ding Liu, Jingwen He, Yang Bai, Fangmin Chen, and Lean Fu. 2022. Residual local feature network for efficient super-resolution. Proceedings of the IEEE/CVF international conference on computer vision. IEEE, New Orleans, LA, USA, 2022, pp. 765-775. https://doi.org/10.1109/CVPRW56347.2022.00092Google ScholarCross Ref
- Zheyuan Li, Yingqi Liu, Xiangyu Chen, Haoming Cai, Jinjin Gu, Yu Qiao, and Chao Dong. 2022. Blueprint separable residual network for efficient image super-resolution. In Proceedings of the IEEE/CVF international conference on computer vision. IEEE, New Orleans, LA, USA, 2022, pp. 832-842. https://doi.org/10.1109/CVPRW56347.2022.00099Google ScholarCross Ref
- Xiaohan Ding, Yuchen Guo, Guiguang Ding, and Jungong Han. 2019. Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks. In Proceedings of the IEEE/CVF international conference on computer vision. IEEE, Seoul, Korea (South), 2019, pp. 1911-1920. https://doi.org/10.1109/ICCV.2019.00200Google ScholarCross Ref
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2017. ImageNet classification with deep convolutional neural networks. Commun. ACM 60, 6 (June 2017), 84–90. https://doi.org/10.1145/3065386Google ScholarDigital Library
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going deeper with convolutions. In Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Boston, MA, USA, 2015, pp. 1-9. https://doi.org/10.1109/CVPR.2015.7298594Google ScholarCross Ref
- Pravendra Singh, Vinay Kumar Verma, Piyush Rai, and Vinay P. Namboodiri. 2019. Hetconv: Heterogeneous kernel-based convolutions for deep cnns. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Long Beach, CA, USA, 2019, pp. 4830-4839. https://doi.org/10.1109/CVPR.2019.00497Google ScholarCross Ref
- Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, Guiguang Ding, and Jian Sun. 2021. Repvgg: Making vgg-style convnets great again. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. IEEE, Nashville, TN, USA, 2021, pp. 13728-13737. https://doi.org/10.1109/CVPR46437.2021.01352Google ScholarCross Ref
- Xiaohan Ding, Xiangyu Zhang, Ningning Ma, Jungong Han, and Guiguang Ding. 2021. Diverse branch block: Building a convolution as an inception-like unit. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, Nashville, TN, USA, 2021, pp. 10881-10890. https://doi.org/10.1109/CVPR46437.2021.01074Google ScholarCross Ref
- Mu Hu, Junyi Feng, Jiashen Hua, Baisheng Lai, Jianqiang Huang, Xiaojin Gong, and Xiansheng Hua. 2022. Online convolutional re-parameterization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, New Orleans, LA, USA, 2022, pp. 558-567. https://doi.org/10.1109/CVPR52688.2022.00065.Google ScholarCross Ref
- Yawei Li, 2022. NTIRE 2022 challenge on efficient super-resolution: Methods and results. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, New Orleans, LA, USA, 2022, pp. 1061-1101. https://doi.org/10.1109/CVPRW56347.2022.00118.Google ScholarCross Ref
- Laurent Sifre, and Stéphane Mallat. 2014. Rigid-motion scattering for texture classification. arXiv:1403.1687. Retrieved from https://arxiv.org/abs/1403.1687Google Scholar
- Daniel Haase, and Manuel Amthor. 2020. Rethinking depthwise separable convolutions: How intra-kernel correlations lead to improved mobilenets. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. IEEE, Seattle, WA, USA, 2020, pp. 14588-14597. https://doi.org/10.1109/CVPR42600.2020.01461Google ScholarCross Ref
- Ying Tai, 2017. Memnet: A persistent memory network for image restoration. In Proceedings of the IEEE international conference on computer vision. IEEE, Venice, Italy, 2017, pp. 4549-4557. https://doi.org/10.1109/ICCV.2017.486Google ScholarCross Ref
- Panqu Wang, Pengfei Chen, Ye Yuan, Ding Liu, Zehua Huang, and Xiaodi Hou; Garrison Cottrell. 2018. Understanding convolution for semantic segmentation. 2018 IEEE winter conference on applications of computer vision (WACV). IEEE, Lake Tahoe, NV, USA, 2018, pp. 1451-1460. https://doi.org/10.1109/WACV.2018.00163Google ScholarCross Ref
- Eirikur Agustsson, and Radu Timofte. 2017. Ntire 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, USA, 2017, pp. 1122-1131. https://doi.org/10.1109/CVPRW.2017.150Google ScholarCross Ref
- Bee Lim, Sanghyun Son, Heewon Kim, Seungjun Nah, and Kyoung Mu Lee. 2017. Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, USA, 2017, pp. 1132-1140. https://doi.org/10.1109/CVPRW.2017.151Google ScholarCross Ref
- Marco Bevilacqua, A. Roumy, Christine Guillemot, and Marie-Line Alberi-Morel. 2012. Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In Proceedings of the British Machine Vision Conference. 2012:135-1. https://doi.org/10.5244/C.26.135Google ScholarCross Ref
- Roman Zeyde, Michael Elad, and Matan Protter. 2012. On single image scale-up using sparse-representations. Curves and Surfaces 2010. Lecture Notes in Computer Science, vol 6920. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-27413-8_47Google ScholarDigital Library
- David Martin, Charless Fowlkes, D. Tal, and Jitendra Malik. 2001. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. IEEE, Vancouver, BC, Canada, 2001, pp. 416-423 vol.2, doi: 10.1109/ICCV.2001.937655Google ScholarCross Ref
- Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja. 2015. Single image super-resolution from transformed self-exemplars. Proceedings of the IEEE conference on computer vision and pattern recognition. IEEE, Boston, MA, USA, 2015, pp. 5197-5206. https://doi.org/10.1109/CVPR.2015.7299156Google ScholarCross Ref
- Abdul Muqeet, Jiwon Hwang, Subin Yang, JungHeum Kang, Yongwoo Kim, and Sung-Ho Bae. 2020. Multi-attention based ultra lightweight image super-resolution. (eds) Computer Vision – ECCV 2020 Workshops. ECCV 2020. Lecture Notes in Computer Science (), vol 12537. Springer, Cham. https://doi.org/10.1007/978-3-030-67070-2_6Google ScholarDigital Library
Index Terms
- Re-parameterizable Residual Multiple Convolutions Network for Efficient Single Image Super-Resolution
Index terms have been assigned to the content through auto-classification.
Recommendations
DCSR: Dilated Convolutions for Single Image Super-Resolution
Dilated convolutions support expanding receptive field without parameter exploration or resolution loss, which turn out to be suitable for pixel-level prediction problems. In this paper, we propose multiscale single image super-resolution (SR) based on ...
Deep ResNet Based Remote Sensing Image Super-Resolution Reconstruction in Discrete Wavelet Domain
AbstractWe present a single-image super-resolution (SR) method for Remote Sensing Image based on deep learning within Discrete Wavelet Domain in this paper. Our method is inspired Residual Learning. Firstly, an input image is decomposed by single level 2D ...
Conditional Convolution Residual Network for Efficient Super-Resolution
Artificial Neural Networks and Machine Learning – ICANN 2023AbstractWith the continuous development of deep learning, single-image super-resolution (SISR) based on convolutional neural networks (CNNs) has made significant progress. Although CNN-based methods have achieved great success, these methods are difficult ...
Comments