Skip to main content
SpringerLink
Log in

Image Super-Resolution Based on the Down-Sampling Iterative Module and Deep CNN

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

Most deep learning-based image SR algorithms do not apply the down-sampling to the reconstructed process. Given this fact and inspired by the iteration idea, we propose a novel image SR method based on the down-sampling iterative module and deep CNN, which explores a new basic iterative module combining up- and down-sampling processes. Each iteration of the iterative module generates the intermediate LR prediction and the HR image. The final reconstructed result is obtained by the weighted summation of the intermediate predicted images generated by multiple iterations. During the training, we adopt the adaptive loss function to achieve fast convergence and accurate reconstruction. Detailed experimental comparisons and analyses show that our method is superior to some state-of-the-art methods in objective performance evaluation and visual effects.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

All data generated or analyzed during this study are included in this published article.

References

  1. A.K. Bhunia, S.R.K. Perla, P. Mukherjee, A. Das, P.P. Roy, Texture synthesis guided deep hashing for texture image retrieval, in 2019 IEEE Winter Conference on Applications of Computer Vision (2019), pp. 609–618

  2. Y.P. Cao, Z.W. He, X. Li, Y.L. Cao, J.X. Yang, Fast and accurate single image super-resolution via an energy-aware improved deep residual network. Sig. Process. 162, 115–125 (2019)

    Article  Google Scholar 

  3. X. Cheng, X. Li, J. Yang, Y. Tai, SESR: single image super resolution with recursive squeeze and excitation networks, in 2018 24th International Conference on Pattern Recognition (2018), pp. 147–152

  4. J. Chu, Z.X. Guo, L. Leng, Object detection based on multi-layer convolution feature fusion and online hard example mining. IEEE Access 6, 19959–19967 (2018)

    Article  Google Scholar 

  5. C. Dong, C.C. Loy, K.M. He, X.O. Tang, Image super-resolution using deep convolutional networks. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 295–307 (2016)

    Article  Google Scholar 

  6. C. Dong, C.C. Loy, X.O. Tang, Accelerating the super-resolution convolutional neural network, in Computer Vision – ECCV (2016), pp. 391–407

  7. Y.C. Fan, H.H. Shi, J.H. Yu, D. Liu, W. Han, H.C. Yu, Z.Y. Wang, X.C. Wang, T.S. Huang, Balanced two-stage residual networks for image super-resolution, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2017), pp. 161–168

  8. S. Gupta, P.P. Roy, D.P. Dogra, B. Kim, Retrieval of colour and texture images using local directional peak valley binary pattern. Pattern Anal. Appl. 23, 1569–1585 (2020)

    Article  Google Scholar 

  9. M. Haris, G. Shakhnarovich, N. Ukita, Deep back-projection networks for single image super-resolution, eprint arXiv (2019). arXiv:1904.05677

  10. J.B. Huang, A. Singh, N. Ahuja, Single image super-resolution from transformed self-exemplars, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 5197–5206

  11. Z. Hui, X.M. Wang, X.B. Gao, Two-stage convolutional network for image super-resolution, in 2018 24th International Conference on Pattern Recognition (2018), pp. 2670–2675

  12. J. Kim, B. Kim, P.P. Roy, D. Jeong, Efficient facial expression recognition algorithm based on hierarchical deep neural network structure. IEEE Access 7, 41273–41285 (2019)

    Article  Google Scholar 

  13. J. Kim, J.K. Lee, K.M. Lee, Accurate image super-resolution using very deep convolutional networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 1646–1654

  14. J. Kim, J.K. Lee, K.M. Lee, Deeply-recursive convolutional network for image super-resolution, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 1637–1645

  15. A. Konwer, A.K. Bhunia, A. Bhowmick, A.K. Bhunia, P. Banerjee, P.P. Roy, U. Pal, Staff line removal using generative adversarial networks, in 2018 24th International Conference on Pattern Recognition (2018), pp. 1103–1108

  16. W.S. Lai, J.B. Huang, N. Ahuja, M.H. Yang, Deep Laplacian pyramid networks for fast and accurate super-resolution, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 624–632

  17. C. Ledig, L. Theis, F. Huszar, J. Caballero, A. Cunningham, A. Acosta, A. Aitken, A. Tejani, J. Totz, Z.H. Wang, W.Z. Shi, Photo-realistic single image super-resolution using a generative adversarial network, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 4681–4690

  18. L. Leng, M. Li, C. Kim, X. Bi, Dual-source discrimination power analysis for multi-instance contactless palmprint recognition. Multimed. Tools Appl. 76, 333–354 (2017)

    Article  Google Scholar 

  19. L. Leng, J.S. Zhang, J. Xu, M.K. Khan, K. Alghathbar, Dynamic weighted discrimination power analysis in DCT domain for face and palmprint recognition, in 2010 International Conference on Information and Communication Technology Convergence (2010), pp. 17–19

  20. Z. Li, J.L. Yang, Z. Liu, X.M. Yang, G. Jeon, W. Wu, Feedback network for image super-resolution, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (2019), pp. 3867–3876

  21. B. Lim, S. Son, H. Kim, S. Nah, K.M. Lee, Enhanced deep residual networks for single image super-resolution, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2017), pp. 136–144

  22. Y. Lu, Y. Zhou, Z.Q. Jiang, X.Q. Guo, Z.X. Yang, Channel attention and multi-level features fusion for single image super-resolution, in 2018 IEEE Visual Communications and Image Processing (2018), pp. 1–4

  23. B. Marco, R. Aline, G. Christine, A. Marie, Low-complexity single-image super-resolution based on nonnegative neighbor embedding, in Proceedings of the 23rd British Machine Vision Conference (2012), pp. 135.1–135.10

  24. A. Mittal, P.P. Roy, P. Singh, B. Raman, Rotation and script independent text detection from video frames using sub pixel mapping. J. Vis. Commun. Image Represent. 46, 187–198 (2017)

    Article  Google Scholar 

  25. O. Ronneberger, P. Fischer, T. Brox, U-net: convolutional networks for biomedical image segmentation, in International Conference on Medical Image Computing and Computer-Assisted Intervention (2015), pp. 234–241

  26. Y. Tai, J. Yang, X.M. Liu, Image super-resolution via deep recursive residual network, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2017), pp. 3147–3155

  27. Y. Tai, J. Yang, X.M. Liu, C.Y. Xu, MemNet: a persistent memory network for image restoration, in Proceedings of the IEEE International Conference on Computer Vision (2017), pp. 4539–4547

  28. R. Timofte, V.D. Smet, L.V. Gool, A+: adjusted anchored neighborhood regression for fast super-resolution, in Asian Conference on Computer Vision (2014), pp. 111–126

  29. T. Tong, G. Li, X.J. Liu, Q.Q. Gao, Image super-resolution using dense skip connections, in Proceedings of the IEEE International Conference on Computer Vision (2017), pp. 4799–4807

  30. Z.W. Wang, D. Liu, J.C. Yang, W. Han, T. Huang, Deep networks for image super-resolution with sparse prior, in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 370–378

  31. X.T. Wang, K. Yu, S.X. Wu, J.J. Gu, Y.H. Liu, C. Dong, Y. Qiao, C.C. Loy, ESRGAN: enhanced super-resolution generative adversarial networks, in Proceedings of the European Conference on Computer Vision Workshops (2018)

  32. J.C. Yang, J. Wright, T. Huang, Y. Ma, Image super-resolution as sparse representation of raw image patches, in 2008 IEEE Conference on Computer Vision and Pattern Recognition (2008), pp. 1–8

  33. Y. Yuan, S.Y. Liu, J.W. Zhang, Y.B. Zhang, C. Dong, L. Lin, Unsupervised image super-resolution using cycle-in-cycle generative adversarial networks, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (2018), pp. 701–710

  34. R. Zeyde, M. Elad, M. Protter, On single image scale-up using sparse-representations, in International Conference on Curves and Surfaces (2010), pp. 711–730

  35. Y.Q. Zhang, J. Chu, L. Leng, J. Miao, Mask-refined R-CNN: a network for refining object details in instance segmentation. Sensors 20(4), 1010 (2020)

    Article  Google Scholar 

  36. Y.L. Zhang, Y.P. Tian, Y. Kong, B.N. Zhong, Y. Fu, Residual dense network for image super-resolution, in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2018), pp. 2472–2481

Download references

Acknowledgements

This research was supported by the National Natural Science Foundation of China (61573182), and by the Fundamental Research Funds for the Central Universities (NS2020025).

Author information

Authors and Affiliations

  1. School of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, People’s Republic of China

    Xin Yang, Yifan Zhang, Tao Li, Yingqing Guo & Dake Zhou

Authors
  1. Xin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yifan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yingqing Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dake Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, X., Zhang, Y., Li, T. et al. Image Super-Resolution Based on the Down-Sampling Iterative Module and Deep CNN. Circuits Syst Signal Process 40, 3437–3455 (2021). https://doi.org/10.1007/s00034-020-01630-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-020-01630-4

Keywords

Search

Navigation