Skip to main content
SpringerLink
Log in

An image super-resolution deep learning network based on multi-level feature extraction module

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Due to the lack of depth of the super-resolution (SR) method based on shallow networks, the feature maps of different convolutional layers have similar receptive fields, so that the performance improvement is not obvious. To solve this problem effectively, we propose an image SR reconstruction deep model based on a new multi-level feature extraction module in this paper. The method constructs an improved multi-level feature extraction module using the dense connection to obtain a deeper network and richer hierarchical feature maps for the SR task. In addition, we apply the loss function combined with the perceptual characteristics to improve the visual effect of the reconstructed image. Experiments show that the proposed method works well at reconstructed images with different magnification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Anwar S, Huynh CP, Porikli F (2017) Chaining identity mapping modules for image denoising[J]. arXiv preprint arXiv:1712.02933

  2. Bai Y, Zhang Y, Ding M, Ghanem B (2018) Sod-mtgan: Small object detection via multi-task generative adversarial network. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 206–221

  3. Bevilacqua M, Roumy A, Guillemot C et al (2012) Low-complexity single-image super-resolution based on nonnegative neighbor[C]. In: Proceedings British Machine Vision Conference, pp 135-1–135.10

  4. Chaudhry AM, Riaz MM, Ghafoor A (2019) Super-resolution based on self-example learning and guided filtering[J]. SIViP 13(2):237–244

    Article  Google Scholar 

  5. Collobert R, Weston J, Bottou L et al (2011) Natural language processing (almost) from scratch[J]. J Mach Learn Res 12(ARTICLE):2493–2537

  6. Dahl GE, Yu D, Deng L, Acero A (2012) Context-dependent pretrained deep neural networks for large-vocabulary speech recognition, IEEE Transactions on audio, speech, and language processing

  7. Dong C, Loy C C, He K et al (2014) Learning a deep convolutional network for image super-resolution[C]. In: European conference on computer vision. Springer, Cham, pp 184–199

  8. Dong C, Loy CC, He K et al (2015) Image super-resolution using deep convolutional networks[J]. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  9. Fanaee F, Yazdi M, Faghihi M (2019) Face image super-resolution via sparse representation and wavelet transform [J]. SIViP 13(1):79–86

    Article  Google Scholar 

  10. Greenspan H (2008) Super-resolution in medical imaging [J]. Comput J 52(1):43–63

    Article  Google Scholar 

  11. He K et al. (2015) Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification”. In: 2015 IEEE International Conference on Computer Vision (ICCV). IEEE

  12. He K, Zhang X, Ren S et al (2016) Deep residual learning for image recognition[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  13. Martin D, Fowlkes C, Tal D et al (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]. In: Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001. IEEE, vol 2, pp 416–423

  14. Huang J B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5197–5206

  15. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars, in: IEEE Conf. Computer Vision Pattern Recognition

  16. Hui Z, Wang X, Gao X (2018) Two-stage convolutional network for image super-resolution[C]. In: 2018 24th International Conference on Pattern Recognition (ICPR). IEEE, pp 2670–2675

  17. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution[C]//European conference on computer vision. Springer, Cham, pp 694–711

    Google Scholar 

  18. Khan S, Rahmani H, Shah SAA, Bennamoun M (2018) A guide to convolutional neural networks for computer vision. Synthesis Lect Comp Vis 8(1):1–207

    Article  Google Scholar 

  19. Kim J, Kwon Lee J, Mu Lee K (2016a) Accurate image super-resolution using very deep convolutional networks[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  20. Kim J, Kwon Lee J, Mu Lee K (2016b) Deeply-recursive convolutional network for image super-resolution[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

  21. Kumar A, Irsoy O, Ondruska P et al (2016) Ask me anything: dynamic memory networks for natural language processing[C]. In: International conference on machine learning, pp 1378–1387

  22. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution, in: IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 624–632

  23. Lai WS, Huang JB, Ahuja N et al (2018) Fast and accurate image super-resolution with deep laplacian pyramid networks[J]. IEEE Trans Pattern Anal Mach Intell 41(11):2599–2613

    Article  Google Scholar 

  24. Lillesand T, Kiefer RW, Chipman J (2015) Remote sensing and image interpretation[M]. John Wiley & Sons, Hoboken

    Google Scholar 

  25. Lim B, Son S, Kim H et al (2017) Enhanced deep residual networks for single image super-resolution[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  26. Mudunuri SP, Biswas S (2015) Low resolution face recognition across variations in pose and illumination [J]. IEEE Trans Pattern Anal Mach Intell 38(5):1034–1040

    Article  Google Scholar 

  27. Prashnani E, Cai H, Mostofi Y, et al. (2018) PieAPP: Perceptual image-error assessment through pairwise preference[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1808-1817

  28. Ren S, He K, Girshick R et al (2015) Faster r-cnn: towards real-time object detection with region proposal networks[C]. In: Advances in neural information processing systems, pp 91–99

  29. Sundar KJA, Vaithiyanathan V (2017) Multi-frame super-resolution using adaptive normalized convolution [J]. SIViP 11(2):357–362

    Article  Google Scholar 

  30. Swaminathan A, Wu M, Liu KJR (2008) Digital image forensics via intrinsic fingerprints[J]. IEEE Trans Inf Forensics Secur 3(1):101–117

  31. Tai Y, Yang J, Liu X et al (2017a) Memnet: a persistent memory network for image restoration[C]. In: Proceedings of the IEEE international conference on computer vision, pp 4539–4547

  32. Tai Y, Yang J, Liu X (2017b) Image super-resolution via deep recursive residual network[C]. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3147–3155

  33. Timofte R, De Smet V, Van Gool L (2014) A+: adjusted anchored neighborhood regression for fast super-resolution[C]. In: Asian conference on computer vision. Springer, Cham, pp 111–126

  34. Timofte R, Agustsson E, Van Gool L et al. (2017) Ntire 2017 challenge on single image super-resolution: Methods and results[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops. 114–125

  35. Tong T, Li G, Liu X et al (2017) Image super-resolution using dense skip connections[C]. In: Proceedings of the IEEE International Conference on Computer Vision, pp 4799–4807

  36. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Loy CC, Qiao Y, Tang X (2018) Esrgan: Enhanced super-resolution generative adversarial networks, arXiv preprint arXiv:1809.00219

  37. Xie C, Liu Y, Zeng W, Lu X (2019) An improved method for single image super-resolution based on deep learning [J]. SIViP 13(3):557–565

    Article  Google Scholar 

  38. Yang S, Wang Z, Zhang L, Wang M (2014) Dual-geometric neighbor embedding for image super resolution with sparse tensor[J]. IEEE Trans Image Process 23(7):2793–2803

    Article  MathSciNet  Google Scholar 

  39. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-rep- resentations, in: International Conference on Curves and Surfaces, pp. 711-730

  40. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. arXiv preprint arXiv:1807.02758.

  41. Zhang Y, Tian Y, Kong Y et al. (2018) Residual dense network for image super-resolution[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2472–2481

  42. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks,” arXiv preprint arXiv:1807.02758

  43. Zhang R, Isola P, Efros A A, et al. (2018) The unreasonable effectiveness of deep features as a perceptual metric[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 586–595

  44. Zhu S, Li Y (2019) Single image super-resolution under multi-frame method [J]. SIViP 13(2):331–339

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

This research was supported by the National Natural Science Foundation of China (61573182) and by the Fundamental Research Funds for the Central Universities (NS2020025).

Author information

Authors and Affiliations

  1. College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 210016, China

    Xin Yang, Yifan Zhang, Yingqing Guo & Dake Zhou

Authors
  1. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yifan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yingqing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dake Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Yang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Zhang, Y., Guo, Y. et al. An image super-resolution deep learning network based on multi-level feature extraction module. Multimed Tools Appl 80, 7063–7075 (2021). https://doi.org/10.1007/s11042-020-09958-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09958-4

Keywords

Search

Navigation