Skip to main content
SpringerLink
Log in

A Novel Fast Reconstruction Method for Single Image Super Resolution Task

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

The in-depth development of generative adversarial networks in the field of specific application tasks, the single image super-resolution problem has been widely studied. We proposed a novel reconstruction model using generative adversarial network for single image super-resolution reconstruction task. Our model directly learns an end-to-end mapping between the low and high-resolution images. The mapping is represented as a generative adversarial network that takes the low-resolution images as the input and outputs the high-resolution one. We further confirmed that the spectral normalization and attention method is exceedingly effective for the training of stably generating adversarial networks. We use a more lightweight external attention method in the network to accelerate the speed of global structure reconstruction. It is different from the previous image super-resolution task. Our model implements multi-scale joint training and optimizes all layers. We explore different networks structures and parameter settings to achieve trade-offs between performance and speed. Quantitative indexes and qualitative results show that our proposed method achieves comparable performance with the state-of-the-art supervised models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. IEEE Conf Comput Vis Pattern Recogn Workshops (CVPRW) 2017:1132–1140. https://doi.org/10.1109/CVPRW.2017.151

    Article  Google Scholar 

  2. Nan F, Zeng Q, Xing Y, Qian Y (2020) Single image super-resolution reconstruction based on the ResNeXt network. Multimed Tools Appl 79:34459–34470. https://doi.org/10.1007/s11042-020-09053-8

    Article  Google Scholar 

  3. Chu J, Li X, Zhang J, Lu W (2020) Super-resolution using multi-channel merged convolutional network. Neurocomputing 394:136–145. https://doi.org/10.1016/j.neucom.2019.04.089

    Article  Google Scholar 

  4. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. IEEE Conf Comput Vis Pattern Recogn (CVPR) 2017:5967–5976. https://doi.org/10.1109/CVPR.2017.632

    Article  Google Scholar 

  5. Wu X, Li Y, Hao Z, Wu C, Wang X, Liu Y (2019) Image style transformation based on structure GAN. Chin Autom Congr (CAC) 2019:2002–2007. https://doi.org/10.1109/CAC48633.2019.8996315

    Article  Google Scholar 

  6. Meng Y, Kong D, Zhu Z, Zhao Y (2019) From night to day: GANs based low quality image enhancement. Neural Process Lett 50:799–814. https://doi.org/10.1007/s11063-018-09968-2

    Article  Google Scholar 

  7. Tian C, Xu Y, Zuo W (2020) Image denoising using deep CNN with batch renormalization. Neural Netw 121:461–473. https://doi.org/10.1016/j.neunet.2019.08.022

    Article  Google Scholar 

  8. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38:295–307. https://doi.org/10.1109/TPAMI.2015.2439281

    Article  Google Scholar 

  9. Lin G, Wu Q, Qiu L, Huang X (2018) Image super-resolution using a dilated convolutional neural network. Neurocomputing 275:1219–1230. https://doi.org/10.1016/j.neucom.2017.09.062

    Article  Google Scholar 

  10. Lai W-S, Huang J-B, Ahuja N, Yang M-H (2017) Deep Laplacian pyramid networks for fast and accurate super-resolution. IEEE Conf Comput Vis Pattern Recogn (CVPR) 2017:5835–5843. https://doi.org/10.1109/CVPR.2017.618

    Article  Google Scholar 

  11. Wang Y, Perazzi F, McWilliams B, Sorkine-Hornung A, Sorkine-Hornung O, Schroers C (2018) A fully progressive approach to single-image super-resolution. IEEE/CVF Conf Comput Vis Pattern Recogn Workshops (CVPRW) 2018:977–97709. https://doi.org/10.1109/CVPRW.2018.00131

    Article  Google Scholar 

  12. Shaham TR, Dekel T, Michaeli T (2019) SinGAN: learning a generative model from a single natural image. IEEE/CVF Int Conf Comput Vis (ICCV) 2019:4569–4579. https://doi.org/10.1109/ICCV.2019.00467

    Article  Google Scholar 

  13. Li Y, Huang H, Zhang L, Wang G, Zhang H, Zhou W (2020) Super-resolution and self-attention with generative adversarial network for improving malignancy characterization of hepatocellular carcinoma. IEEE 17th International Symposium on Biomedical Imaging (ISBI), 2020, pp 1556–1560

  14. Seddik MEA, Tamaazousti M, Lin J (2020) Generative collaborative networks for single image super-resolution. Neurocomputing 398:293–303. https://doi.org/10.1016/j.neucom.2019.02.068

    Article  Google Scholar 

  15. Zhao M, Liu X, Liu H, Wong KKL (2020) Super-resolution of cardiac magnetic resonance images using Laplacian Pyramid based on Generative Adversarial Networks. Comput Med Imaging Graph 80:101698. https://doi.org/10.1016/j.compmedimag.2020.101698

    Article  Google Scholar 

  16. Yu L, Long X, Tong C (2018) Single image super-resolution based on improved WGAN. Atlantis Press, pp 101–104

    Google Scholar 

  17. Zong L, Chen L (2019) Single image super-resolution based on self-attention. IEEE Int Conf Unmanned Syst Artif Intell (ICUSAI) 2019:56–60. https://doi.org/10.1109/ICUSAI47366.2019.9124791

    Article  Google Scholar 

  18. Wang M, Chen Z, Wu QMJ, Jian M (2020) Improved face super-resolution generative adversarial networks. Mach Vis Appl 31:22. https://doi.org/10.1007/s00138-020-01073-6

    Article  Google Scholar 

  19. Zhang H, Xu T, Li H, Zhang S, Wang X, Huang X et al (2019) StackGAN++: realistic image synthesis with stacked generative adversarial networks. IEEE Trans Pattern Anal Mach Intell 41:1947–1962. https://doi.org/10.1109/TPAMI.2018.2856256

    Article  Google Scholar 

  20. Zhang H, Xu T, Li H, Zhang S, Wang X, Huang X et al (2017) StackGAN: text to photo-realistic image synthesis with stacked generative adversarial networks. IEEE Int Conf Comput Vis (ICCV) 2017:5908–5916. https://doi.org/10.1109/ICCV.2017.629

    Article  Google Scholar 

  21. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C et al (2019) ESRGAN: enhanced super-resolution generative adversarial networks. In: Leal-Taixé L, Roth S (eds) Computer Vision—ECCV 2018 Workshops. Springer, Cham, pp 63–79

    Chapter  Google Scholar 

  22. Hinz T, Fisher M, Wang O, et al. (2021) Improved techniques for training single-image GANs[A]. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)[C]. pp. 1299–1308

  23. Du J, Cheng K, Yu Y, Wang D, Zhou H (2020) Panchromatic image super-resolution via self attention-augmented WGAN 2020

  24. Guo M-H, Liu Z-N, Mu T-J, Hu S-M (2021) Beyond self-attention: external attention using two linear layers for visual tasks. arXiv 2021. arXiv preprint arXiv:2105.02358

  25. Xiaopeng C, Jiangzhong C, Yuqin L, Qingyun D (2020) Improved training of spectral normalization generative adversarial networks. 2020 2nd World Symposium on Artificial Intelligence (WSAI), pp 24–28

  26. Lan Z, Lin M, Li X, Hauptmann AG, Raj B (2015) Beyond Gaussian pyramid: multi-skip feature stacking for action recognition. IEEE Conf Comput Vis Pattern Recogn (CVPR) 2015:204–212. https://doi.org/10.1109/CVPR.2015.7298616

    Article  Google Scholar 

  27. Burt P, Adelson E (1983) The Laplacian pyramid as a compact image code. IEEE Trans Commun 31:532–540. https://doi.org/10.1109/TCOM.1983.1095851

    Article  Google Scholar 

  28. Paris S, Hasinoff SW, Kautz J (2011) Local Laplacian filters: edge-aware image processing with a Laplacian pyramid. ACM Trans Graph 30:1–12. https://doi.org/10.1145/2010324.1964963

    Article  Google Scholar 

  29. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. IEEE/CVF Conf Comput Vis Pattern Recogn 2018:7794–7803. https://doi.org/10.1109/CVPR.2018.00813

    Article  Google Scholar 

  30. Garbin C, Zhu X, Marques O (2020) Dropout vs. batch normalization: an empirical study of their impact to deep learning. Multimed Tools Appl 79:12777–12815. https://doi.org/10.1007/s11042-019-08453-9

    Article  Google Scholar 

  31. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A (2017) Improved training of Wasserstein GANs. Adv Neural Inf Process Syst (NeurIPS) 2017:5767–5777

    Google Scholar 

  32. Chen X, Zhao H, Yang D, Li Y, Kang Q, Lu H (2021) SA-SinGAN: self-attention for single-image generation adversarial networks. Mach Vis Appl 32:104. https://doi.org/10.1007/s00138-021-01228-z

    Article  Google Scholar 

  33. Peng D, Yang W, Liu C, Lü S (2021) SAM-GAN: self-attention supporting multi-stage generative adversarial networks for text-to-image synthesis. Neural Netw 138:57–67. https://doi.org/10.1016/j.neunet.2021.01.023

    Article  Google Scholar 

  34. Karras T, Aila T, Laine S, Lehtinen J (2018) Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196

  35. Obukhov A, Krasnyanskiy M (2020) Quality assessment method for GAN based on modified metrics inception score and Fréchet Inception distance. In: Silhavy R, Silhavy P, Prokopova Z (eds) Software engineering perspectives in intelligent systems. Springer, Cham, pp 102–114

    Chapter  Google Scholar 

  36. Erfurt J, Helmrich CR, Bosse S, Schwarz H, Marpe D, Wiegand T (2019) A study of the perceptually weighted peak signal-to-noise ratio (WPSNR) for image compression. IEEE Int Conf Image Process (ICIP) 2019:2339–2343. https://doi.org/10.1109/ICIP.2019.8803307

    Article  Google Scholar 

  37. Ćalasan M, Abdel Aleem SHE, Zobaa AF (2020) On the root mean square error (RMSE) calculation for parameter estimation of photovoltaic models: a novel exact analytical solution based on Lambert W function. Energy Convers Manag 210:112716. https://doi.org/10.1016/j.enconman.2020.112716

    Article  Google Scholar 

  38. Zhang L, Zhang L, Bovik AC (2015) A feature-enriched completely blind image quality evaluator. IEEE Trans Image Process 24:2579–2591. https://doi.org/10.1109/TIP.2015.2426416

    Article  MathSciNet  MATH  Google Scholar 

  39. Lempitsky V, Vedaldi A, Ulyanov D (2018) Deep image prior. In: IEEE Computer Society

  40. Shocher A, Cohen N, Irani M (2018) Zero-shot super-resolution using deep internal learning. In: IEEE Computer Society

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Hebei University of Technology, Tianjin, 300401, China

    Xi Chen & Hongdong Zhao

Authors
  1. Xi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongdong Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hongdong Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, X., Zhao, H. A Novel Fast Reconstruction Method for Single Image Super Resolution Task. Neural Process Lett 55, 9995–10010 (2023). https://doi.org/10.1007/s11063-023-11235-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11235-y

Keywords

Search

Navigation