Skip to main content
SpringerLink
Log in

s-LMPNet: a super-lightweight multi-stage progressive network for image super-resolution

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Single image super-resolution (SISR) has achieved great success in recent years due to the representation ability of large and deep models. However, these models usually have a large number of network parameters, which hinders their application to real-world scenarios. To reduce the number of parameters in the SISR models, we propose a super-lightweight model termed s-LMPNet with a multi-stage architecture. Specifically, s-LMPNet includes three sub-networks, which are organized in a cascaded way. Each sub-network is constructed with multiple lightweight cross-group skip-connecting blocks (CGSCBs). To enhance the model performance, a residual feature fusion attention module is adopted to integrate intermediate features from different CGSCBs in a self-adaptive weighted way. A cross-stage feature propagation module is used to propagate the information from the low stage to the high stage, thereby making the network optimization procedure more stable. Extensive experiments are performed on commonly-used super-resolution benchmarks. Experiment results have shown that s-LMPNet achieves promising performance compared to other state-of-the-art lightweight super-resolution methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Du X (2022) Single image super-resolution using global enhanced upscale network. Appl Intell 52(3):2813–2819

    Article  Google Scholar 

  2. Yan Y, Liu C, Chen C, Sun X, Jin L, Peng X, Zhou X (2021) Fine-grained attention and feature-sharing generative adversarial networks for single image super-resolution. IEEE Trans Multimedia 24:1473–1487

    Article  Google Scholar 

  3. Chen W, Yao P, Gai S, Da F (2022) Multi-scale feature aggregation network for image super-resolution. Appl Intell 52(4):3577–3586

    Article  Google Scholar 

  4. Hu Y, Li J, Huang Y, Gao X (2021) Image super-resolution with self-similarity prior guided network and sample-discriminating learning. IEEE Trans Circuits Syst Video Technol 32(4):1966–1985

    Article  Google Scholar 

  5. Wan J, Yin H, Chong A-X, Liu Z-H (2020) Progressive residual networks for image super-resolution. Appl Intell 50(5):1620–1632

    Article  Google Scholar 

  6. Zhang J, Long C, Wang Y, Piao H, Mei H, Yang X, Yin B (2021) A two-stage attentive network for single image super-resolution. IEEE Trans Circuits Syst Video Technol 32(3):1020–1033

    Article  Google Scholar 

  7. Chen Y, Liu L, Phonevilay V, Gu K, Xia R, Xie J, Zhang Q, Yang K (2021) Image super-resolution reconstruction based on feature map attention mechanism. Appl Intell 51(7):4367–4380

    Article  Google Scholar 

  8. Zhang D, Zhu B, Zhong Y (2022) Mbmr-net: multi-branches multi-resolution cross-projection network for single image super-resolution. Appl Intell:1–15

  9. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Proceedings of the European conference on computer vision, pp 184–199

  10. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision, pp 286–301

  11. Guo Y, Chen J, Wang J, Chen Q, Cao J, Deng Z, Xu Y, Tan M (2020) Closed-loop matters: dual regression networks for single image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5407–5416

  12. Yang A, Yang B, Ji Z, Pang Y, Shao L (2020) Lightweight group convolutional network for single image super-resolution. Inf Sci 516:220–233

    Article  Google Scholar 

  13. Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the European conference on computer vision, pp 252–268

  14. Yang W, Wang W, Zhang X, Sun S, Liao Q (2019) Lightweight feature fusion network for single image super-resolution. IEEE Signal Process Lett 26(4):538–542

    Article  Google Scholar 

  15. Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 723–731

  16. Hui Z, Gao X, Yang Y, Wang X (2019) Lightweight image super-resolution with information multi-distillation network. In: Proceedings of the 27th ACM international conference on multimedia, pp 2024–2032

  17. Luo X, Xie Y, Zhang Y, Qu Y, Li C, Fu Y (2020) Latticenet: towards lightweight image super-resolution with lattice block. In: Proceedings of the European conference on computer vision, pp 272–289

  18. Wang L, Dong X, Wang Y, Ying X, Lin Z, An W, Guo Y (2021) Exploring sparsity in image super-resolution for efficient inference. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4917–4926

  19. Ren D, Zuo W, Hu Q, Zhu P, Meng D (2019) Progressive image deraining networks: a better and simpler baseline. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3937–3946

  20. Fu X, Liang B, Huang Y, Ding X, Paisley J (2019) Lightweight pyramid networks for image deraining. IEEE Trans Neural Netw Learn Syst 31(6):1794–1807

    Article  Google Scholar 

  21. Li X, Wu J, Lin Z, Liu H, Zha H (2018) Recurrent squeeze-and-excitation context aggregation net for single image deraining. In: Proceedings of the European conference on computer vision, pp 254–269

  22. Zheng Y, Yu X, Liu M, Zhang S (2019) Residual multiscale based single image deraining. In: British machine vision conference, p 147

  23. Suin M, Purohit K, Rajagopalan A (2020) Spatially-attentive patch-hierarchical network for adaptive motion deblurring. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3606–3615

  24. Zhang H, Dai Y, Li H, Koniusz P (2019) Deep stacked hierarchical multi-patch network for image deblurring. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5978–5986

  25. Zamir SW, Arora A, Khan S, Hayat M, Khan FS, Yang M-H, Shao L (2021) Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14821–14831

  26. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Proceedings of the European conference on computer vision, pp 391–407

  27. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  28. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3147–3155

  29. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1874–1883

  30. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L.-C (2018) Mobilenetv2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4510–4520

  31. Li X, Wang W, Hu X, Yang J (2019) Selective kernel networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 510–519

  32. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Change Loy C (2018) Esrgan: enhanced super-resolution generative adversarial networks. In: Proceedings of the European conference on computer vision workshops, pp 0–0

  33. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 126–135

  34. Du J, Wei W, Fan C, Zou L, Shen J, Zhou Z, Chen Z (2020) Lightweight image super-resolution with mobile share-source network. IEEE Access 8:60008–60018

    Article  Google Scholar 

  35. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the British machine vision conference, pp 1–10

  36. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces, pp 711–730

  37. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings eighth IEEE international conference on computer vision, vol 2, pp 416–423

  38. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–5206

  39. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimed Tools Appl 76(20):21811–21838

    Article  Google Scholar 

  40. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  41. Wang C, Li Z, Shi J (2019) Lightweight image super-resolution with adaptive weighted learning network. arXiv:1904.02358

  42. Li Z, Yang J, Liu Z, Yang X, Jeon G, Wu W (2019) Feedback network for image super-resolution. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3867–3876

  43. Chu X, Zhang B, Ma H, Xu R, Li Q (2021) Fast, accurate and lightweight super-resolution with neural architecture search. In: 2020 25th international conference on pattern recognition, pp 59–64

  44. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

  45. Karras T, Laine S, Aila T (2019) A style-based generator architecture for generative adversarial networks. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4401–4410

  46. Yang L, Wang S, Ma S, Gao W, Liu C, Wang P, Ren P (2020) Hifacegan: face renovation via collaborative suppression and replenishment. In: Proceedings of the 28th ACM international conference on multimedia, pp 1551–1560

  47. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681– 4690

  48. Huang H, He R, Sun Z, Tan T (2017) Wavelet-srnet: a wavelet-based cnn for multi-scale face super resolution. In: Proceedings of the IEEE international conference on computer vision, pp 1689–1697

  49. Bulat A, Tzimiropoulos G (2018) Super-fan: integrated facial landmark localization and super-resolution of real-world low resolution faces in arbitrary poses with gans. In: Proceedings of the IEEE Conference on computer vision and pattern recognition, pp 109–117

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grants 62072042.

Author information

Authors and Affiliations

  1. School of Computer Science, Beijing Institute of Technology, Beijing, China

    Meng Li, Bo Ma, Ying Liu & Yulin Zhang

Authors
  1. Meng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bo Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ying Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yulin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Meng Li.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, M., Ma, B., Liu, Y. et al. s-LMPNet: a super-lightweight multi-stage progressive network for image super-resolution. Appl Intell 53, 13378–13397 (2023). https://doi.org/10.1007/s10489-022-04185-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04185-w

Keywords

Search

Navigation