Skip to main content
SpringerLink
Log in

Bi-path network coupling for single image super-resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Recent researches have shown that deep convolutional neural networks can significantly boost the performance of single-image super-resolution (SISR). In particular, residual network and densely convolutional network can improve performance remarkably. The residual network enables feature re-usage and the dense skip connections enables new features exploration, which are both favor for feature extraction. In order to alleviate the vanishing-gradient problem in very deep convolution networks. In this paper, a bi-path network coupling is presented for SISR by combining the residual network and the dense skip connections in a very deep network. More specifically, the feature maps in the proposed network are split into two paths, one path is propagated in the form of residual connections, and another is propagated by dense skip connections. In addition, we input the feature maps obtained from the two paths into the coupling layer for feature fusion. Finally, the deconvolution layers are integrated into the network to upscale the feature map for significantly accelerating the network, that the mapping is learned from the low-resolution image to the high-resolution image directly. The proposed network was evaluated on four benchmark datasets and has achieved competing or even higher peak signal-to-noise ratio (PSNR) than most of state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Bevilacqua M, Roumy A, Guillemot C, AlberiMorel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding. In BMVC

  2. Bevilacqua M, Roumy A, Guillemot C, Morel M-LA (2014) Single-image superresolution via linear mapping of interpolated self-examples. IEEE Trans Image Process 23(12):5334–5347

    Article  MathSciNet  MATH  Google Scholar 

  3. Blu T, Thevenaz P, Unser M (2004) Linear interpolation revitalized. IEEE Trans Image Process 13(6):710–719

    Article  MathSciNet  Google Scholar 

  4. Chen Y, Li J, Xiao H et al. (2017) Dual path networks[C]//Advances in Neural Information Processing Systems. 4470-4478

  5. Dai S, Han M, Xu W, Wu Y, Gong Y, Katsaggelos AK (2009) SoftCuts: A soft edge smoothness prior for color image super-resolution. IEEE Trans Image Process 18(5):969–981

    Article  MathSciNet  MATH  Google Scholar 

  6. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database, Computer Vision and Pattern Recognition 2009. CVPR 2009. IEEE Conference on, pp. 248-255

  7. Dong C, Loy CC, He K, et al. (2014) Learning a deep convolutional network for image super-resolution[C]//European Conference on Computer Vision. Springer, Cham, 184-199

  8. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network[C]//European Conference on Computer Vision. Springer, Cham, 391-407

  9. He K, Zhang X, Ren S et al. (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification[C]//Proceedings of the IEEE international conference on computer vision. 1026-1034

  10. He K, Zhang X, Ren S et al. (2016) Deep residual learning for image recognition[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 770-778

  11. Huang J, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.

  12. Huang G, Liu Z, Weinberger KQ et al (2017) Densely connected convolutional networks[C]. Proc IEEE Conf Comput Vis Pattern Recognit 1(2):3

    Google Scholar 

  13. Hung KW, Siu WC (2012) Robust soft-decision interpolation using weighted least squares. IEEE Trans Image Process 21(3):1061–1069

    Article  MathSciNet  MATH  Google Scholar 

  14. Hung KW, Siu WC (2012) Single image super-resolution using iterative Wiener filter, in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), Kyoto, Japan, Mar. 1269-1272

  15. Karras T, Aila T, Laine S, Lehtinen J (2017) Progressive growing of gans for improved quality, stability, and variation. submitted to ICLR 2018

  16. Kim J, Kwon LJ, Mu LK (2016) Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1646-1654

  17. Kim J, Kwon LJ, Mu LK (2016) Deeply-recursive convolutional network for image super-resolution[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 1637-1645

  18. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization[J]. arXiv preprint arXiv:1412.6980.

  19. Lai WS, Huang JB, Ahuja N, et al. (2017) Deep laplacian pyramid networks for fast and accurate super-resolution[C]//Proc. IEEE Conf. Comput. Vis. Pattern Recognit. 624-632

  20. Ledig C, Theis L, Husz` ar F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z et al. (2016) Photo-realistic single image super-resolution using a generative adversarial network. arXiv:1609.04802

  21. Lim B, Son S, Kim H et al. (2017) Enhanced deep residual networks for single image super-resolution[C]//The IEEE conference on computer vision and pattern recognition (CVPR) workshops. 1(2): 4.

  22. Liu H, Feng Y, Li L (2010) Multi-channel fast super-resolution image reconstruction based on matrix observation model[J]. J Harbin Inst Techn 17(2):239–246

    Google Scholar 

  23. Mao X, Shen C, Yang Y-B Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In NIPS 2016

  24. Marquina A, Osher SJ (2008) Image super-resolution by TV-regularization and Bregman iteration. J Sci Comput 37(3):367–382

    Article  MathSciNet  MATH  Google Scholar 

  25. Martin D, Fowlkes C, Tal D et al. (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics[C]//Computer Vision, 2001. ICCV 2001. Proceedings. Eighth IEEE International Conference on. IEEE, 2: 416-423

  26. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation[C]//International Conference on Medical image computing and computer-assisted intervention. Springer, Cham, 234-241

  27. Schulter S, Leistner C, Bischof H (2015) Fast and accurate image upscaling with super-resolution forests[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 3791-3799.

  28. Shi W, Caballero J, Ledig C, Zhuang X, Bai W, Bhatia K, Marvao AMSM de, Dawes T, ORegan D, Rueckert D (2013) Cardiac image super-resolution with global correspondence using multi-atlas patchmatch. In MICCAI

  29. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In CVPR

  30. Tai Y, Yang J, Liu X et al. (2017) Memnet: A persistent memory network for image restoration[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 4539-4547

  31. Timofte R, De SV, Van GL (2014) A+: Adjusted anchored neighborhood regression for fast super-resolution[C]//Asian Conference on Computer Vision. Springer, Cham, 111-126

  32. Timofte R, Agustsson E, Van Gool L, Yang M-H, Zhang L, Lim B, Son S, Kim H, Nah S, Lee KM et al. (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In CVPRW

  33. Tong T, Li G, Liu X et al. (2017) Image Super-Resolution Using Dense Skip Connections[C]//2017 IEEE International Conference on Computer Vision (ICCV). IEEE, 4809-4817

  34. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  35. Wang Y, Liu Q, Zhou H, Wang Y (2018) Learning multi-denoising autoencoding priors for image super-resolution. J Vis Commun Image Represent 57:152–162

    Article  Google Scholar 

  36. Xu R, Zhang J, Huang K (2016) Image super-resolution using two-channel convolutional neural networks[J]. J Image Graph 21(5):556–564

    Google Scholar 

  37. Yang CY, Yang MH (2013) Fast direct super-resolution by simple functions[C]//Computer Vision (ICCV), 2013 IEEE International Conference on. IEEE, 561-568.

  38. Yang J, Wright J, Huang T, et al. (2008) Image super-resolution as sparse representation of raw image patches[C]//Proc. IEEE Conf. on Computer Vision and Pattern Recognition. 1-8

  39. Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation[J]. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  40. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In Proceedings of the International Conference on Curves and Surfaces

  41. Zhang K, Zuo W, Zhang L (2018) Learning a single convolutional super-resolution network for multiple degradations, in Proc. IEEE Conf. Comput. Vis. Pattern Recog.

  42. Zhao Y, Wang RG, Jia W, Wang W-M, Gao W (2017) Iterative projection reconstruction for fast and efficient image upsampling. Neurocomputing 226:200–211

    Article  Google Scholar 

  43. Zou W, Yuen PC (2012) Very low resolution face recognition problem[J]. IEEE Trans Image Process 21(1):327–340

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The authors sincerely thank the anonymous reviewers for their valuable comments and constructive suggestions that are very helpful in the improvement of this paper. This work was supported in part by the National Natural Science Foundation of China under 61661028, 61871206, 61661031, 61463035, and the Natural Science Foundation of Jiangxi Province (20181BAB202003).

Author information

Authors and Affiliations

  1. School of Electronic Information Engineering, Nanchang University, Nanchang, 330031, China

    Yalin Yang, Qiegen Liu, Minghui Zhang & Yuhao Wang

Authors
  1. Yalin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qiegen Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minghui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuhao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuhao Wang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, Y., Liu, Q., Zhang, M. et al. Bi-path network coupling for single image super-resolution. Multimed Tools Appl 78, 21981–21998 (2019). https://doi.org/10.1007/s11042-019-7511-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7511-x

Keywords

Search

Navigation