Skip to main content

Advertisement

Springer Nature Link
Log in

Conditional generative adversarial network with densely-connected residual learning for single image super-resolution

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

Recently, generative adversarial network (GAN) has been widely employed in single image super-resolution (SISR), achieving favorably good perceptual effects. However, the SR outputs generated by GAN still have some fictitious details, which are quite different from the ground-truth images, resulting in a low PSNR value. In this paper, we leverage the ground-truth high-resolution (HR) image as a useful guide to learn an effective conditional GAN (CGAN) for SISR. Among it, we design the generator network via residual learning, which introduces dense connections to the residual blocks to effectively fuse low and high-level features across different layers. Extensive evaluations show that our proposed SR method performs much better than state-of-the-art methods in terms of PSNR, SSIM, and visual perception.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Ahn N, Kang B, Sohn K-A (2018) Fast, accurate, and lightweight super-resolution with cascading residual network. In: Proceedings of the european conference on computer vision (ECCV), pp 252–268

  2. Bevilacqua M, Roumy A, Guillemot C, Alberi-Morel ML (2012) Low-complexity single-image super-resolution based on nonnegative neighbor embedding

  3. Dong C, Loy CC, He K, Tang X (2014) Learning a deep convolutional network for image super-resolution. In: European conference on computer vision. Springer, pp 184–199

  4. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. In: European conference on computer vision, pp 391–407. Springer

  5. Fan D-P, Cheng M-M, Liu J-J, Gao S-H, Hou Q, Borji A (2018) Salient objects in clutter: bringing salient object detection to the foreground. In: European conference on computer vision (ECCV). Springer

  6. Fu K, Zhao Q, Gu IY, Yang J (2019) Deepside: a general deep framework for salient object detection. Neurocomputing 356:69–82

    Article  Google Scholar 

  7. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  8. He X, Mo Z, Wang P, Liu Y, Yang M, Cheng J (2019) Ode-inspired network design for single image super-resolution. In: 2019 IEEE conference on computer vision and pattern recognition

  9. He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision, pp 1026–1034

  10. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  11. He K, Zhang X, Ren S, Sun J (2016) Identity mappings in deep residual networks. In: European conference on computer vision, pp 630–645. Springer

  12. Huang G, Liu Z, Maaten LVD, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  13. Huang J-B, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–5206

  14. Hui Z, Wang X, Gao X (2018) Fast and accurate single image super-resolution via information distillation network. In: CVPR, pp 723–731

  15. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167

  16. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  17. Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: European conference on computer vision, pp 694–711. Springer

  18. Kim J, Lee JK, Lee KM (2016) Deeply-recursive convolutional network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1637–1645

  19. Kim J, Lee JK, Lee KM (2016) Accurate image super-resolution using very deep convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1646–1654

  20. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 624–632

  21. Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, et al. (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  22. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  23. Martin D, Fowlkes C, Tal D, Malik J, et al. (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Iccv Vancouver

  24. Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv:1411.1784

  25. Qiao J, Song H, Zhang K, Zhang X, Liu Q (2019) Image super-resolution using conditional generative adversarial network. IET Image Process 13:2673–2679

    Article  Google Scholar 

  26. Radford A, Metz L, Chintala S (2015) Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv:1511.06434

  27. Shi W, Caballero J, Huszár F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1874–1883

  28. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  29. Tian C, Xu Y, Zuo W, Zhang B, Fei L, Lin C (2020) Coarse-to-fine cnn for image super-resolution. IEEE Trans Multimed, pp 1–1

  30. Timofte R, De Smet V, Van Gool L (2014) A+: adjusted anchored neighborhood regression for fast super-resolution. In: Asian conference on computer vision, pp 111–126. Springer

  31. Tong T, Li G, Liu X, Gao Q (2017) Image super-resolution using dense skip connections. In: Proceedings of the IEEE international conference on computer vision, pp 4799–4807

  32. Vella M, Mota JFC (2019) Single image super-resolution via CNN architectures and TV-TV minimization. CoRR, arXiv:1907.05380

  33. Wang Y, Wang L, Wangb H, Li P (2019) End-to-end image super-resolution via deep and shallow convolutional networks. IEEE Access, pp 1–1

  34. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Yu Q, Loy CC (2018) Esrgan: enhanced super-resolution generative adversarial networks. In: European conference on computer vision, pp 63–79. Springer

  35. Yang C-Y, Ma C, Yang M-H (2014) Single-image super-resolution: a benchmark. In: European conference on computer vision, pp 372–386. Springer

  36. Yang X, Mei H, Zhang J, Xu K, Yin B, Zhang Q, Wei X (2019) Drfn: deep recurrent fusion network for single-image super-resolution with large factors. IEEE Trans Multimed 21(2):328–337

    Article  Google Scholar 

  37. Yang Q, Yang R, Davis J, Nistér D (2007) Spatial-depth super resolution for range images. In: 2007 IEEE conference on computer vision and pattern recognition, pp 1–8. IEEE

  38. Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: International conference on curves and surfaces, pp 711–730. Springer

  39. Zhang Y, Tian Y, Yu K, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2472–2481

  40. Zhao J-X, Liu J-J, Fan D-P, Cao Y, Yang J, Cheng M-M (2019) Egnet:edge guidance network for salient object detection. In: IEEE international conference on computer vision

Download references

Acknowledgements

This work is supported in part by National Major Project of China for New Generation of AI (No. 2018AAA0100400), in part by the NSFC (61872189, 61876088), in part by the NSF of Jiangsu Province (BK20191397, BK20170040), in part by the 333 High-level Talents Cultivation Project of Jiangsu Province (BRA2020291).

Author information

Authors and Affiliations

  1. Jiangsu Key Laboratory of Big Data Analysis Technology (B-DAT) and Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science & Technology, Nanjing, China

    Jiaojiao Qiao, Huihui Song, Kaihua Zhang & Xiaolu Zhang

Authors
  1. Jiaojiao Qiao
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Huihui Song
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Kaihua Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xiaolu Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Kaihua Zhang.

Ethics declarations

Conflict of interests

The authors declare that they have no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiao, J., Song, H., Zhang, K. et al. Conditional generative adversarial network with densely-connected residual learning for single image super-resolution. Multimed Tools Appl 80, 4383–4397 (2021). https://doi.org/10.1007/s11042-020-09817-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09817-2

Keywords