Skip to main content
SpringerLink
Log in

Deep quantification down-plain-upsampling residual learning for single image super-resolution

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Deep convolutional neural networks have been widely used in single image super-resolution (SISR) with great success. However, the performance and efficiency of such models need to be improved for practical applications. In this paper, a novel deep quantification down-plain-upsampling (QDPU) network for SISR is proposed. In the framework, a down-plain-upsampling (DPU) residual block based on U-Net is firstly designed to reduce the computational cost by transforming the spatial scale of feature maps without sacrificing the reconstruction performance. Then, to better transmit low-level features to the reconstruction layer, we construct quantification skip-connection modules to integrate the outputs of the DPU residual blocks. Finally, QDPU is developed by stacking the DPU residual blocks with multiple skip-connections to reconstruct high-resolution images and reduce the computational burden. Quantitative and qualitative evaluations of the reconstruction results on four benchmark datasets show that the proposed method can achieve better performance compared with several state-of-the-art SISR methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Nasrollahi K, Moeslund TB (2014) Super-resolution: a comprehensive survey. Mach Vis Appl 25(6):1423–1468

    Article  Google Scholar 

  2. Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527

    Article  Google Scholar 

  3. Zhang L, Wu X (2006) An edge-guided image interpolation algorithm via directional filtering and data fusion. IEEE Trans Image Process 15(8):2226–2238

    Article  Google Scholar 

  4. Protter M, Elad M, Takeda H, Milanfar P (2008) Generalizing the Nonlocal-means to super-resolution reconstruction. IEEE Trans Image Process 18(1):36–51

    Article  MathSciNet  MATH  Google Scholar 

  5. Sun J, Xu Z, Shum HY (2008) Image super-resolution using gradient profile prior. In: IEEE conference on computer vision and pattern recognition. IEEE, Anchorage, AK, pp 1–8

    Google Scholar 

  6. Sun J, Xu Z, Shum HY (2011) Gradient profile prior and its applications in image super-resolution and enhancement. IEEE Trans Image Process 20(6):1529

    Article  MathSciNet  MATH  Google Scholar 

  7. Tai YW, Liu S, Brown MS, Lin S (2010) Super resolution using edge prior and single image detail synthesis. In: IEEE conference on computer vision and pattern recognition. IEEE, San Francisco, CA, pp 2400–2407

    Google Scholar 

  8. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  9. Huang S, Sun J, Yang Y, Fang Y, Lin P, Que Y (2019) Robust single-image super-resolution based on adaptive edge-preserving smoothing regularization. IEEE Trans Image Process 27(6):2650

    Article  MathSciNet  MATH  Google Scholar 

  10. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 105–114

    Google Scholar 

  11. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1646–1654

    Google Scholar 

  12. Yang W, Feng J, Yang J, Zhao F, Liu J, Guo Z, Yan S (2017) Deep edge guided recurrent residual learning for image super-resolution. IEEE Trans Image Process 26(12):5895–5907

    Article  MathSciNet  Google Scholar 

  13. Park SC, Min KP, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36

    Article  Google Scholar 

  14. Huang H, He R, Sun Z, Tan T (2017) Wavelet-SRNet: a wavelet-based CNN for multi-scale face super resolution. In: IEEE conference on computer vision. IEEE, Venice, pp 1698–1706

    Google Scholar 

  15. Dong C, Chen CL, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  16. Guo T, Mousavi HS, Vu TH, Monga V (2017) Deep wavelet prediction for image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1100–1109

    Google Scholar 

  17. Lai WS, Huang JB, Ahuja N, Yang MH (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 5835–5843

    Google Scholar 

  18. Zhang Y, Sun L, Yan C, Ji X, Dai Q (2018) Adaptive residual networks for high-quality image restoration. IEEE Trans Image Process 99:1

    MathSciNet  MATH  Google Scholar 

  19. Dong C, Chen CL, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision. Amsterdam, pp 391–407

  20. Shi W, Caballero J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1874–1883

    Google Scholar 

  21. Gregor K, Lecun Y (2010) Learning fast approximations of sparse coding. In: International conference on international conference on machine learning. Haifa, Israel, pp 399–406

  22. Wang Z, Liu D, Yang J, Han W, Huang T (2015) Deep networks for image super-resolution with sparse prior. In: IEEE international conference on computer vision. IEEE, Santiago, Chile, pp 370–378

    Google Scholar 

  23. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 1637–1645

    Google Scholar 

  24. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations. San Diego, CA, pp 1–14

  25. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference on computer vision and pattern recognition. IEEE, Las Vegas, NV, pp 770–778

    Google Scholar 

  26. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1132–1140

    Google Scholar 

  27. Timofte R, Agustsson E, Gool LV, Yang MH, Zhang L, Lim B, Son S, Kim H, Nah S, Lee KM (2017) NTIRE 2017 challenge on single image super-resolution: methods and results. In: IEEE conference on computer vision and pattern recognition workshops. IEEE, Honolulu, HI, pp 1110–1121

    Google Scholar 

  28. Tai Y, Yang J, Liu X (2017) Image super-resolution via deep recursive residual network. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 2790–2798

    Google Scholar 

  29. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, pp 2472–2481

    Google Scholar 

  30. Srivastava RK, Greff K, Schmidhuber J (2015) Highway networks. arXiv:1505.00387

  31. Iandola FN, Han S, Moskewicz MW, Ashraf K, Dally WJ, Keutzer K (2016) SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and < 0.5 MB model size. arXiv:1602.07360

  32. Krizhevsky A, Sutskever I, Hinton GE (2012) ImageNet classification with deep convolutional neural networks. In: International conference on neural information processing systems. Lake Tahoe, NV, pp 1097–1105

  33. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer assisted intervention. Munich, pp 234–241

  34. Iglovikov V, Shvets A (2018) TernausNet: U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. arXiv:1801.05746

  35. Lin M, Chen Q, Yan S (2014) Network in network. In: International conference on learning representations. Banff, Canada, pp 1–10

  36. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: IEEE conference on computer vision and pattern recognition. IEEE, Salt Lake City, UT, pp 7132–7141

    Google Scholar 

  37. Tai Y, Yang J, Liu X, Xu C (2017) MemNet: a persistent memory network for image restoration. In: IEEE conference on computer vision. IEEE, Venice, Italy, pp 4549–4557

    Google Scholar 

  38. Kamnitsas K, Ledig C, Newcombe VFJ, Simpson JP, Kane AD, Menon DK, Rueckert D, Glocker B (2016) Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Med Image Anal 36:61

    Article  Google Scholar 

  39. Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder–decoder architecture for scene segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  40. Nah S, Kim TH, Lee KM (2017) Deep multi-scale convolutional neural network for dynamic scene deblurring. In: IEEE conference on computer vision and pattern recognition. IEEE, Honolulu, HI, pp 257–265

    Google Scholar 

  41. Odena A, Dumoulin V, Olah C (2016) Deconvolution and checkerboard artifacts. Distill. https://doi.org/10.23915/distill.00003

    Article  Google Scholar 

  42. Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2017) Inception-v4, Inception-ResNet and the impact of residual connections on learning. In: AAAI conference on artificial intelligence. San Francisco, CA, pp 4278–4284

  43. Zeyde R, Elad M, Protter M (2012) On single image scale-up using sparse-representations. In: International conference on curves and surfaces. Avignon, France, pp 711–730

  44. Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980

  45. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on ImageNet classification. In: IEEE international conference on computer vision. IEEE, Santiago, Chile, pp 1026–1034

    Google Scholar 

  46. Martin DR, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE international conference on computer vision. IEEE, Vancouver, BC, pp 416–423

    Google Scholar 

  47. Agustsson E, Timofte R (2017) NTIRE 2017 challenge on single image super-resolution: dataset and study. In: IEEE conference on computer vision pattern and recognition workshops. IEEE, Honolulu, HI, pp 1122–1131

    Google Scholar 

  48. Bevilacqua M, Roumy A, Guillemot C, Morel A (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In: Electronic proceedings of the British machine vision conference. Guildford, Surrey, pp 1–10

  49. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: IEEE conference on computer vision and pattern recognition. IEEE, Boston, MA, pp 5197–5206

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China (No. 61862030, No. 61662026, and No. 61462031), by the Natural Science Foundation of Jiangxi Province (No. 20182BCB22006, No. 20181BAB202010, No. 20192ACB20002, and No. 20192ACBL21008), and by the Project of the Education Department of Jiangxi Province (No. KJLD14031, No. GJJ170312, and No. GJJ170318).

Author information

Authors and Affiliations

  1. School of Software and Communication Engineering, Jiangxi University of Finance and Economics, Nanchang, 330032, China

    Shuying Huang, Haijun Zhu & Yingjun Tang

  2. School of Information Technology, Jiangxi University of Finance and Economics, Nanchang, 330032, China

    Yong Yang & Yifan Zuo

Authors
  1. Shuying Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Haijun Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yifan Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yingjun Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Yang.

Ethics declarations

Conflict of interest

The authors declare no conflict of interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, S., Zhu, H., Yang, Y. et al. Deep quantification down-plain-upsampling residual learning for single image super-resolution. Int. J. Mach. Learn. & Cyber. 11, 1923–1937 (2020). https://doi.org/10.1007/s13042-020-01083-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01083-w

Keywords

Search

Navigation