Skip to main content
SpringerLink
Log in

Lightweight image super-resolution with feature cheap convolution and attention mechanism

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

Since deep learning is introduced into the field of super-resolution (SR), many deep learning-based SR methods have been proposed and achieved good results. At present, most neural networks use ordinary convolution and deeper neural layer in image super-resolution reconstruction in order to achieve better results. Therefore, most existing models have a massive amount of parameters and calculation, which limits the practical application. To solve this problem, in this paper, we propose an efficient feature cheap convolution SR (FCCSR) model, which consists of several multi-level information fusion Blocks (MIFB). In the MIFB, the features are extracted by combining normal convolution and cheap convolution, and the features obtained from each layer of convolution are fused. Under limited parameters and reconstruction speed, it enables the whole model to obtain the features from shallow to deep layers in the low-resolution (LR) images. Secondly, a novel lightweight channel attention module is designed to obtain fewer parameters and better performance. Finally, we add a gradient loss function to the original L1 loss function, making the model pay more attention to the high-frequency part of the LR image and making the reconstructed image’s texture details more straightforward. The experimental results show that the proposed model has reached the most advanced level regarding image quality, memory consumption, parameter number, and calculation amount. The operation speed of FCCSR is roughly the same as that of the current STOA models, and the number of parameters is only 0.63 M. On some data sets, the best performance of FCCSR can be improved by 7%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

All data, models, and code generated or used during the study appear in the submitted article.

References

  1. Yang X, Liu D, Zhou D, et al. (2018) Super-resolution reconstruction of face images based on pre-amplification non-negative restricted neighborhood embedding[J]. Bulletin of the Polish Academy of Sciences. Technical Sciences 66(6)

  2. Zhu, X., Li, Z., Lou, J., et al.: Video super-resolution based on a spatio-temporal matching network. Pattern Recogn. 110, 107619 (2020)

    Article  Google Scholar 

  3. Al-Janabi, S., Alkaim, A.F., Adel, Z.: An Innovative synthesis of deep learning techniques (DCapsNet & DCOM) for generation electrical renewable energy from wind energy[J]. Soft. Comput. 24(14), 10943–10962 (2020)

    Article  Google Scholar 

  4. Dong, Chao, et al. (2014) Learning a deep convolutional network for image super-resolution European conference on computer vision. Springer Cham

  5. Lim Bee et al. (2017) Enhanced deep residual networks for single image super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition workshops

  6. Zhang, Yulun, et al. (2018) Residual dense network for image super-resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  7. Zhang, Yulun, et al. (2018) Image super-resolution using very deep residual channel attention networks. Proceedings of the European Conference on Computer Vision (ECCV)

  8. Dai, Tao, et al. (2019) Second-order Attention Network for Single Image Super-Resolution. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

  9. Ledig, Christian, et al. (2017) Photo-realistic single image super-resolution using a generative adversarial network. Proceedings of the IEEE conference on computer vision and pattern recognition

  10. Xintao Wang, Ke Yu, Shixiang Wu, Jinjin Gu, Yihao Liu, Chao Dong, Yu Qiao, and Chen Change Loy. Esrgan: Enhanced super-resolution generative adversarial networks. In ECCV, pages 63–79. Springer, 2018

  11. Ma C , Rao Y , Cheng Y , et al. (2020) Structure-Preserving Super Resolution with Gradient Guidance

  12. Han, K., Wang, Y., Tian, Q., et al.: GhostNet More features from cheap operations[C]. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2020, 1580–1589 (2020)

    Google Scholar 

  13. Szegedy, C., Vanhoucke, V., Ioffe, S., et al.: Rethinking the inception architecture for computer vision[C]. Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 2016, 2818–2826 (2016)

    Google Scholar 

  14. Kim, Jiwon, Jung Kwon Lee, and Kyoung Mu Lee (2016) Accurate image super-resolution using very deep convolutional networks. Proceedings of the IEEE conference on computer vision and pattern recognition

  15. Kim, Jiwon, Jung Kwon Lee, and Kyoung Mu Lee. (2016) Deeply-recursive convolutional network for image super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition

  16. Lai, Wei-Sheng, et al. (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. Proceedings of the IEEE conference on computer vision and pattern recognition

  17. ZhengHui, X., meiWang, and XinboGao,: Fast and accurate single image super-resolution via information distillation network. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018, 723–731 (2018)

    Google Scholar 

  18. Huang, G., Liu, Z., Weinberger, K.Q.: Densely connected convolutional networks. CoRR 1608, 06993 (2016)

    Google Scholar 

  19. Namhyuk Ahn,Byungkon Kang, and Kyung-AhSohn.2018. Fast, Accurate, and Lightweight Super-Resolution with Cascading ResidualNetwork.InEuropean Conference on Computer Vision (ECCV).252–268

  20. Hui Z, Gao X, Yang Y, et al. (2019) Lightweight image super-resolution with information multi-distillation network[C]. Proceedings of the 27th ACM International Conference on Multimedia 2024–2032

  21. Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556, 2014. 1, 7

  22. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. Imagenet classification with deep convolutional neural networks. Bartlettetal 48: 1106–1114

  23. Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam Mobilenets: Efficient convolutional neural networks for mobile vision applications. CoRR, abs/1704.04861, 2017

  24. Ding, X., Guo, Y., Ding, G., et al.: Acnet: Strengthening the kernel skeletons for powerful cnn via asymmetric convolution blocks[C]. Proceedings of the IEEE International Conference on Computer Vision. 2019, 1911–1920 (2019)

    Google Scholar 

  25. Hwang, J.W., Lee, H.S.: Adaptive image interpolation based on local gradient features[J]. IEEE Signal Process. Lett. 11(3), 359–362 (2004)

    Article  Google Scholar 

  26. Shi, Wenzhe, et al. (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network." Proceedings of the IEEE conference on computer vision and pattern recognition

  27. Jie Hu, Li Shen (2018) Samule Albanie and Gang Sun. Squeeze-and-Excitation Networks. In CVPR

  28. Abualigah, L., Diabat, A., Sumari, P., et al.: Applications, deployments, and integration of internet of drones (iod) a review[J]. IEEE Sensors J 21, 2553 (2021)

    Article  Google Scholar 

  29. Roman Zeyde, Michael Elad, and MatanProtter (2010) On single image scale-up using sparse-representations. In International conference on curves and surfaces.Springer 711–730

  30. Abualigah, L., Yousri, D., Abd Elaziz, M., et al.: Aquila optimizer: a novel meta-heuristic optimization algorithm[J]. Comput. Ind. Eng. 157, 107250 (2021)

    Article  Google Scholar 

  31. Jia-Bin Huang, Abhishek Singh, and Narendra Ahuja (2015) Single image super-resolution from transformed self-exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition 5197–5206

  32. Matsui, Y., Ito, K., Aramaki, Y., Fujimoto, A., Ogawa, T., Yamasaki, T., Aizawa, K.: Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications 76(20), 21811–21838 (2017)

    Article  Google Scholar 

  33. Dong, C., et al.: Image super-resolution using deep convolutional networks. IEEE Transac Pattern Anal Mac Intell 382, 295–307 (2015)

    Google Scholar 

  34. YingTai, JianYang, and XiaomingLiu (2017) Image super-resolution via deep recursive residual network In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 3147–3155

  35. Tai, Ying, et al. (2017) Memnet: A persistent memory network for image restoration. Proceedings of the IEEE international conference on computer vision

  36. Zhang K, Zuo W, Zhang L (2018) Learning a Single Convolutional Super-Resolution Network for Multiple Degradations[C]. IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE,

  37. Abualigah, L., Abd Elaziz, M., Sumari, P., et al.: Reptile Search Algorithm (RSA): A nature-inspired meta-heuristic optimizer[J]. Expert Syst. Appl. 191, 116158 (2022)

    Article  Google Scholar 

  38. Yu F, Koltun V. Multi-scale context aggregation by dilated convolutions[J]. arXiv preprint arXiv:1511.07122, 2015.

  39. Al-Janabi, S., Alkaim, A.F.: A nifty collaborative analysis to predicting a novel tool (DRFLLS) for missing values estimation[J]. Soft. Comput. 24(1), 555–569 (2020)

    Article  Google Scholar 

  40. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Transac. Pattern Analys. Mac. Intell. 33(5), 898–916 (2011)

    Article  Google Scholar 

  41. Abualigah, L., Diabat, A., Mirjalili, S., et al.: The arithmetic optimization algorithm[J]. Comput. Methods Appl. Mech. Eng. 376, 113609 (2021)

    Article  MathSciNet  MATH  Google Scholar 

  42. Al-Janabi, S., Alkaim, A., Al-Janabi, E., et al.: Intelligent forecaster of concentrations (PM2. 5, PM10, NO2, CO, O3, SO2) caused air pollution (IFCsAP)[J]. Neural Comput. Appl. 33(21), 14199–14229 (2021)

    Article  Google Scholar 

  43. Jie, L., Tang, J., Gangshan, Wu.: Feature distillation network for lightweight image super-resolution. Springer International Publishing, Cham (2021)

    Google Scholar 

  44. Bevilacqua, M., Roumy, A., Guillemot, C., Alberi-Morel, M.L.: Low-complexity single-image super-resolution based on nonnegative neighbor embedding. Neural Comput Applic 33, 14199 (2012)

    Google Scholar 

  45. Al-Janabi S, Alkaim A (2022) A novel optimization algorithm (Lion-AYAD) to find optimal DNA protein synthesis[J]. Egyptian Informatics Journal

  46. Muqeet, A., et al.: Multi-Attention Based Ultra Lightweight Image Super-Resolution. Springer International Publishing, Cham (2021)

    Google Scholar 

  47. Al-Janabi, S., Mohammad, M., Al-Sultan, A.: A new method for prediction of air pollution based on intelligent computation[J]. Soft. Comput. 24(1), 661–680 (2020)

    Article  Google Scholar 

  48. Mahdi, M.A., AlJanabi, S.: A novel software to improve healthcare base on predictive analytics and mobile services for cloud data centers[C]. International conference on big data and networks technologies. Springer, Cham 2019, 320–339 (2019)

    Google Scholar 

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (61573182, 62073164), and by the Fundamental Research Funds for the Central Universities (NS2022041, NS2020025).

Author information

Authors and Affiliations

  1. College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China

    Xin Yang, Hengrui Li & Xiaochuan Li

Authors
  1. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hengrui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaochuan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XY contributed to the conception of the study, performed the data analyses and wrote the manuscript; HL performed the experiment, and helped perform the analysis with constructive discussions; XL contributed significantly to analysis and manuscript preparation.

Corresponding author

Correspondence to Xin Yang.

Ethics declarations

Conflict of interest

All authors declare that they have no conflict of interest.

Ethical approval

I certify that this manuscript is original and has not been published and will not be submitted elsewhere for publication while being considered by “Cluster Computing”. And the study is not split up into several parts to increase the quantity of submissions and submitted to various journals or to one journal over time. No data have been fabricated or manipulated (including images) to support your conclusions. No data, text, or theories by others are presented as if they were our own. The submission has been received explicitly from all co-authors. And authors whose names appear on the submission have contributed sufficiently to the scientific work and therefore share collective responsibility and accountability for the results.

Informed consent

None.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Li, H. & Li, X. Lightweight image super-resolution with feature cheap convolution and attention mechanism. Cluster Comput 25, 3977–3992 (2022). https://doi.org/10.1007/s10586-022-03631-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-022-03631-1

Keywords

Search

Navigation