Skip to main content
SpringerLink
Log in

Multi-Scale Image Super-Resolution via Hierarchical Filter Groups

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The development of deep convolutional neural networks (DNN) has progressed the recent research interest on super-resolution (SR). The existing DNNs benefited from residual learning and achieved improved performance. In this paper, we propose a discrete wavelet-based hierarchical filter groups (HFG) for multi-scale image SR. Discrete wavelet transform (DWT) produces four sub-bands with different frequency components. Three detail sub-bands (i.e., horizontal, vertical and diagonal) contain sparse wavelet coefficients whereas, approximation sub-band contains average spatial image information. The reconstruction accuracy is improved by exploiting the information in all the wavelet sub-bands simultaneously. Furthermore, hierarchical filter groups are adopted for creating computationally efficient DNN without compromising performance. Moreover, symmetrical skip connections are used for feature reuse in different layers of DNN to improve reconstruction accuracy. In addition, the problem of vanishing of gradients and feature redundancy are mitigated by short skip paths created using symmetrical skip connections. The proposed HFG method achieves superior performance over the other recent competitive methods on benchmark datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ahn N, Kang B, Sohn KA (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision – ECCV, vol 2018. Springer International Publishing, Cham, pp 256–272

  2. Amaranageswarao G, Deivalakshmi S, Ko SB (2020) Residual learning based densely connected deep dilated network for joint deblocking and super resolution. Applied Intelligence https://doi.org/10.1007/s10489-020-01670-y

  3. Amaranageswarao, G, Deivalakshmi, S, Ko, SB (2020) Wavelet based medical image super resolution using cross connected residual-in-dense grouped convolutional neural network. Journal of Visual Communication and Image Representation 70:102819, https://doi.org/10.1016/j.jvcir.2020.102819, http://www.sciencedirect.com/science/article/pii/S1047320320300699

  4. Bevilacqua, M, Roumy, A, Guillemot, C, line Alberi Morel M (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Proceedings of the British Machine Vision Conference, BMVA Press, pp 135.1–135.10, https://doi.org/10.5244/C.26.135

  5. Choi J, Kim M (2017) A deep convolutional neural network with selection units for super-resolution. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW), pp 1150–1156, https://doi.org/10.1109/CVPRW.2017.153

  6. Dong, C, Loy, CC, He, K, Tang, X (2016) Image super-resolution using deep convolutional networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 38(2):295–307, https://doi.org/10.1109/TPAMI.2015.2439281

  7. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision – ECCV, vol 2016. Springer International Publishing, Cham, pp 391–407

  8. Guo T, Mousavi HS, Vu TH, Monga V (2017) Deep wavelet prediction for image super-resolution. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW), pp 1100–1109, https://doi.org/10.1109/CVPRW.2017.148

  9. Huang G, Liu Z, v d Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 2261–2269, https://doi.org/10.1109/CVPR.2017.243

  10. Huang, J, Singh, A, Ahuja, N (2015) Single image super-resolution from transformed self-exemplars. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 5197–5206, https://doi.org/10.1109/CVPR.2015.7299156

  11. Ioannou Y, Robertson D, Cipolla R, Criminisi A (2017) Deep roots: Improving cnn efficiency with hierarchical filter groups. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 5977–5986, https://doi.org/10.1109/CVPR.2017.633

  12. Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of GANs for improved quality, stability and variation. In: International Conference on Learning Representations

  13. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 1646–1654, https://doi.org/10.1109/CVPR.2016.182

  14. Kingma, D, Ba, J (2014) Adam: A method for stochastic optimization. International Conference on Learning Representations

  15. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges C J C, Bottou L, Weinberger K Q (eds) Advances in neural information processing systems, vol 25. Inc., Curran Associates, pp 1097–1105

  16. Kumar N, Verma R, Sethi A (2017) Convolutional Neural networks for wavelet domain super resolution. Pattern Recognition Letters 90. https://doi.org/10.1016/j.patrec.2017.03.014

  17. Lai W, Huang J, Ahuja N, Yang M (2019) Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Transactions on Pattern Analysis and Machine Intelligence 41(11): 2599–2613. https://doi.org/10.1109/TPAMI.2018.2865304

  18. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 105–114, https://doi.org/10.1109/CVPR.2017.19

  19. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW), pp 1132–1140, https://doi.org/10.1109/CVPRW.2017.151

  20. Liu J, Zhang W, Tang Y, Tang J, Wu G (2020) Residual feature aggregation network for image super-resolution. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 2356–2365, https://doi.org/10.1109/CVPR.42600.2020.00243

  21. Liu P, Zhang H, Zhang K, Lin L, Zuo W (2018) Multi-level wavelet-cnn for image restoration. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition workshops (CVPRW), pp 886–88609, https://doi.org/10.1109/CVPRW.2018.00121

  22. Mao XJ, Shen C, Yang Y (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: NIPS

  23. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001, vol 2, pp 416–423 vol.2, https://doi.org/10.1109/ICCV.2001.937655

  24. Matsui Y, Ito K, Aramaki Y, Fujimoto A, Ogawa T, Yamasaki T, Aizawa K (2017) Sketch-based manga retrieval using manga109 dataset. Multimedia Tools and Applications 76(20):21811–21838, https://doi.org/10.1007/s11042-016-4020-z

  25. Mei Y, Fan Y, Zhou Y, Huang L, Huang TS, Shi H (2020) Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining. In: 2020 IEEE/CVF Conference on computer vision and pattern recognition (CVPR), pp 5689–5698, https://doi.org/10.1109/CVPR.42600.2020.00573

  26. Sajjadi MSM, Schölkopf B, Hirsch M (2017) Enhancenet: Single image super-resolution through automated texture synthesis. In: Proceedings IEEE International Conference on Computer Vision (ICCV), IEEE, Piscataway. NJ, USA, pp 4501–4510

  27. Sheikh HR, Bovik AC, de Veciana G (2005) An information fidelity criterion for image quality assessment using natural scene statistics. IEEE Trans Image Process 14(12):2117–2128

    Article  Google Scholar 

  28. Shi W, Caballero J, Ledig C, Zhuang X, Bai W, Bhatia K, de Marvao AMSM, Dawes T, O’Regan D, Rueckert D (2013) Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In: Mori K, Sakuma I, Sato Y, Barillot C, Navab N (eds) Medical image computing and computer-assisted intervention – MICCAI, vol 2013. Springer Berlin Heidelberg, Berlin, Heidelberg, pp 9–16

  29. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pp 2790–2798, https://doi.org/10.1109/CVPR.2017.298

  30. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: 2017 IEEE International conference on computer vision (ICCV), pp 4549–4557, https://doi.org/10.1109/ICCV.2017.486

  31. Timofte R, Agustsson E, Gool LV, Yang M, Zhang L, Lim B, Son S, Kim H, Nah S, Lee KM, Wang X, Tian Y, Yu K, Zhang Y, Wu S, Dong C, Lin L, Qiao Y, Loy CC, Bae W, Yoo J, Han Y, Ye JC, Choi J, Kim M, Fan Y, Yu J, Han W, Liu D, Yu H, Wang Z, Shi H, Wang X, Huang TS, Chen Y, Zhang K, Zuo W, Tang Z, Luo L, Li S, Fu M, Cao L, Heng W, Bui G, Le T, Duan Y, Tao D, Wang R, Lin X, Pang J, Xu J, Zhao Y, Xu X, Pan J, Sun D, Zhang Y, Song X, Dai Y, Qin X, Huynh X, Guo T, Mousavi HS, Vu TH, Monga V, Cruz C, Egiazarian K, Katkovnik V, Mehta R, Jain AK, Agarwalla A, Praveen CVS, Zhou R, Wen H, Zhu C, Xia Z, Wang Z, Guo Q (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW), pp 1110–1121, https://doi.org/10.1109/CVPRW.2017.149

  32. Wei D, Li YM (2019) Convolution and multichannel sampling for the offset linear canonical transform and their applications. IEEE Transactions on Signal Processing 67(23):6009–6024. https://doi.org/10.1109/TSP.2019.2951191

  33. Wei, D, Yang, W, Li, YM (2019) Lattices sampling and sampling rate conversion of multidimensional bandlimited signals in the linear canonical transform domain. Journal of the Franklin Institute 356(13):7571–7607, https://doi.org/10.1016/j.jfranklin.2019.06.031, https://www.sciencedirect.com/science/article/pii/S0016003219304788https://www.sciencedirect.com/science/article/pii/S0016003219304788

  34. Yu, J, Fan, Y, Yang, J, Xu, N, Wang, Z, Wang, X, Huang, TS (2018) Wide activation for efficient and accurate image super-resolution. CoRR abs/1808.08718

  35. Yu X, Ye X, Gao Q (2020) Infrared handprint image restoration algorithm based on apoptotic mechanism. IEEE Access 8:47334–47343. https://doi.org/10.1109/ACCESS.2020.2979018

  36. Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: Chenin P, Cohen A, Gout C, Lyche T, Mazure M L, Schumaker L (eds) Boissonnat JD. Curves and Surfaces, Springer Berlin Heidelberg, Berlin, Heidelberg, pp 711–730

  37. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing 26(7):3142–3155. https://doi.org/10.1109/TIP.2017.2662206

  38. Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 3262–3271, https://doi.org/10.1109/CVPR.2018.00344

  39. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Ferrari V, Hebert M, Sminchisescu C, Weiss Y (eds) Computer vision – ECCV, vol 2018. Springer International Publishing, Cham, pp 294–310

  40. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition, pp 2472–2481, https://doi.org/10.1109/CVPR.2018.00262

  41. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

  42. Zhu, H, Zhang, Q, Wang, Q, Li, H (2020) 4d light field superpixel and segmentation. IEEE Transactions on Image Processing 29:85–99, https://doi.org/10.1109/TIP.2019.2927330

  43. Zou WWW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Transactions on Image Processing 21(1):327–340. https://doi.org/10.1109/TIP.2011.2162423

Download references

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology, Thiruchirappalli, India

    Gadipudi Amaranageswarao & S. Deivalakshmi

Authors
  1. Gadipudi Amaranageswarao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. Deivalakshmi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gadipudi Amaranageswarao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amaranageswarao, G., Deivalakshmi, S. Multi-Scale Image Super-Resolution via Hierarchical Filter Groups. Appl Intell 52, 7550–7565 (2022). https://doi.org/10.1007/s10489-021-02832-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02832-2

Keywords

Search

Navigation