Skip to main content
SpringerLink
Log in

Frequency separation-based multi-scale cascading residual block network for image super resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Deep Convolutional Neural Network (CNN) has recently obtained remarkable achievements in single image super-resolution (SISR). Whereas, these existing methods are usually associated with abundant parameters or computational complexity, which highly limits the real-time application. To solve this problem, we propose a lightweight network named FSCRNet. In general, the proposed network consists of three parts: division schema, feature extraction block, and reconstruction block. Specifically, we decouple the image into two parts: content features and detail features, and then perform different operations separately. Concretely, for detailed features, by combining multi-scale strategy and cascading residual block (MSCRB), the model can explore features and propagate messages efficiently. Also, we introduce channel attention to enhance high-frequency feature representation ability. We use a content feature module (CFM) for content features, consisting of asymmetric convolutions to fetch the tensor elements from the horizontal and vertical directions. We demonstrate that the proposed method with few parameters performs favorably on the benchmarks in quantitative and qualitative results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Alkanha L, Alotaibi D, Albrahim N, Alrayes S and Bchir O (2020) Super-resolution using deep learning to support person identification in surveillance video. International Journal of Advanced Computer Ence and Applications 11(7)

  2. Arbelez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence 33(5):898–916

    Article  Google Scholar 

  3. Bevilacqua M, Roumy A, Guillemot C, Morel MA (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the British machine vision conference. BMVA Press, pp 135.1–135.10

  4. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. CoRR, vol abs/1608.00367

  5. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: Fleet D, Pajdla T, Schiele B, Tuytelaars T (eds) Computer vision – ECCV 2014. Springer International Publishing, Cham, pp 184–199

  6. Gao S, Cheng M, Zhao K, Zhang X, Yang M, Torr PHS (2019) Res2net: a new multi-scale backbone architecture. CoRR, vol abs/1904.01169

  7. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. CoRR, vol abs/1512.03385

  8. Huang J, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE conference on computer vision and pattern recognition, CVPR 2015. IEEE Computer Society, Boston, MA, USA, pp 5197–5206

  9. Hu J, Shen L, Albanie S, Sun G, Wu E (2020) Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 42(8):2011–2023

    Article  Google Scholar 

  10. Izonin I, Tkachenko R, Peleshko D, Rak T, Batyuk D (2015) Learning-based image super-resolution using weight coefficients of synaptic connections. In: 2015 Xth international scientific and technical conference. Computer Sciences and Information Technologies (CSIT), pp 25–29

  11. Kim J, Lee JK, Lee KM (2015) Accurate image super-resolution using very deep convolutional networks. CoRR, vol abs/1511.04587, pp 1646–1654

  12. Kim J, Lee JK, Lee KM (2015) Deeply-recursive convolutional network for image super-resolution. CoRR, vol abs/1511.04491

  13. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Bengio Y, LeCun Y (eds) 3rd international conference on learning representations, ICLR 2015, San Diego, CA, USA. Conference Track Proceedings

  14. Lai W, Huang J, Ahuja N, Yang M (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. CoRR, vol abs/1704.03915

  15. Ledig C, Theis L, Huszar F, Caballero J, Cunningham A, Acosta A, Aitken AP, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017. IEEE Computer Society, Honolulu, HI, USA, pp 105–114

  16. Li J, Fang F, Mei K, Zhang G (2018) Multi-scale residual network for image super-resolution. In: Proceedings of the European conference on computer vision (ECCV), pp 517–532

  17. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. CoRR, vol abs/1707.02921

  18. Liu S, Huang D, Wang Y (2018) Receptive field block net for accurate and fast object detection. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision - ECCV 2018 - 15th European conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XI, vol 11215 of Lecture Notes in Computer Science. Springer, pp 404–419

  19. Lu X, Ma C, Ni B, Yang X, Reid ID, Yang M-H (2018) Deep regression tracking with shrinkage loss. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision - ECCV 2018 - 15th European Conference, Munich, Germany, September 8-14, 2018, Proceedings, Part XIV. Lecture Notes in Computer Science

  20. Lu X, Wang W, Danelljan M, Zhou T, Shen J, Van Gool L (2020) Video object segmentation with episodic graph memory networks. In: Vedaldi A, Bischof H, Brox T, Frahm J-M (eds) Computer vision - ECCV 2020 - 16th European conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III, vol 12348 of Lecture Notes in Computer Science. Springer, pp 661–679

  21. Lu X, Wang W, Ma C, Shen J, Shao L, Porikli F (2019) See more, know more: unsupervised video object segmentation with co-attention siamese networks. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA

  22. Peleshko D, Rak T, Peleshko M, Izonin I, Batyuk D (2016) Two-frames image superresolution based on the aggregate divergence matrix. In: 2016 IEEE first international conference on data stream mining and processing (DSMP). IEEE, pp 235–238

  23. Peleshko D, Rak T, Izonin I (2016) Image superresolution via divergence matrix automatic detection of crossover. International Journal of Intelligent Systems and Applications 8(12):1

    Article  Google Scholar 

  24. Rashkevych Y, Peleshko D, Vynokurova O, Izonin I, Lotoshynska N (2017) Single-frame image super-resolution based on singular square matrix operator. In: 2017 IEEE first Ukraine conference on electrical and computer engineering (UKRCON). IEEE, pp 944–948

  25. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein MS et al (2015) Imagenet large scale visual recognition challenge. International Journal of Computer Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  26. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. CoRR, vol abs/1609.05158

  27. Shi W, Jiang F, Zhao D (2017) Single image super-resolution with dilated convolution based multi-scale information learning inception module. CoRR, vol abs/1707.07128

  28. Song X, Dai Y, Zhou D, Liu L, Li W, Li H, Yang R (2020) Channel attention based iterative residual learning for depth map super-resolution. In: 2020 IEEE/CVF conference on computer vision and pattern recognition, CVPR 2020, Seattle, WA, USA. IEEE, pp 5630–5639

  29. Song T, Chowdhury SR, Yang F, Dutta J (2020) Pet image super-resolution using generative adversarial networks. Neural Networks 125:83–91

    Article  Google Scholar 

  30. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: 2017 IEEE conference on computer vision and pattern recognition, CVPR 2017, Honolulu, HI, USA. IEEE Computer Society, pp 2790–2798

  31. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. CoRR, vol abs/1708.02209

  32. Timofte R, De Smet V, Van Gool L (2015) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Cremers D, Reid I, Saito H, Yang M-H (eds) Computer vision – ACCV 2014. Springer International Publishing, Cham, pp 111–126

  33. Tkachenko R, Tkachenko P, Izonin I, Tsymbal Y (2018) Learning-based image scaling using neural-like structure of geometric transformation paradigm. In: Advances in soft computing and machine learning in image processing. Springer, pp 537–565

  34. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4):600–612

    Article  Google Scholar 

  35. Ye C, Evanusa M, He H, Mitrokhin A, Goldstein T, Yorke JA, Fermüller C, Aloimonos Y (2019) Network deconvolution. CoRR, vol abs/1905.11926,

  36. Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Boissonnat J-D, Chenin P, Cohen A, Gout C, Lyche T, Mazure M-L, Schumaker L (eds) Curves and surfaces. Springer, Berlin, pp 711–730

  37. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. CoRR, vol abs/1807.02758

  38. Zhang M, Liu Z, He J, Yu L (2019) Crnet: image super-resolution using a convolutional sparse coding inspired network, arXiv: Image and Video Processing

  39. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. CoRR, vol abs/1802.08797

  40. Zhang Y, Zong R, Han J, Zhang D, Rashid T, Wang D (2020) Transres: a deep transfer learning approach to migratable image super-resolution in remote urban sensing. In: 2020 17th annual IEEE international conference on sensing, communication, and networking (SECON), pp 1–9

  41. Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep cnn denoiser prior for image restoration. CoRR, vol abs/1704.03264

Download references

Acknowledgements

This work was supported in part by the Innovation Project of National Natural Science Foundation of China under grants (61866009), Guet Graduate Education(2020YCXS055).

Author information

Authors and Affiliations

  1. 541004, GuiLin, China

    Zhenbing Liu, Lu Yuan & Long Sun

Authors
  1. Zhenbing Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lu Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Long Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenbing Liu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, Z., Yuan, L. & Sun, L. Frequency separation-based multi-scale cascading residual block network for image super resolution. Multimed Tools Appl 81, 6827–6848 (2022). https://doi.org/10.1007/s11042-021-11724-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11724-z

Keywords

Search

Navigation