Skip to main content
SpringerLink
Log in

Deeper super-resolution generative adversarial network with gradient penalty for sonar image enhancement

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In the field of underwater wireless communication, the acoustic signal has the advantages of low attenuation, long propagation distance, and high fidelity compared with electromagnetic wave signals. Therefore, as an important acoustic sensor, the sonar has been widely used in underwater topographical surveying, underwater search and rescue, ship navigation, etc. The sonar image also faces challenge of low-resolution due to its imaging mechanism, like ultrasonic image and synthetic aperture radar image. In this paper, we propose a deeper super-resolution generative adversarial network (DGP-SRGAN) with gradient penalty. It can be used to produce the sonar image with high-resolution. The main contribution of our method is that the gradient penalty is added to the loss function for a more stable and faster training network. The deep of the generator network is doubled from the original 16 layers to 32 layers to make the network more expressive, achieving its better performance. The loss function of the discriminator network increases the gradient penalty term. It can insure a faster network converge and then reach a stable state in less time. Thus the proposed network model achieves a better super-resolution reconstruction effect. The experimental results show that DGP-SRGAN can control the output of super-resolution images well based on input conditions. Meanwhile, the quality of the output image has improved significantly when compared with the other methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Chen Z, Tong Y (2017) Face super-resolution through Wasserstein GANs. arXiv preprint arXiv:1705.02438

  2. Dong C, Loy C., He K, Tang X (2015) Image super-resolution using deep convolutional network. IEEE Trans Pattern Anal Machine Intel 38 (2):295–307

    Article  Google Scholar 

  3. Dong C, Loy CC, Tang X (2016) Accelerating the super-resolution convolutional neural network. European conference on computer vision, pp 391–407

  4. Elad M, Feuer A (1997) Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images. IEEE Trans Image Process 6(12):1646–1658

    Article  Google Scholar 

  5. Gao Z, Zhang H, Dong S, Sun S, Wang X, Yang G, de Albuquerque VHC (2020) Salient object detection in the distributed cloud-edge intelligent network. IEEE Netw 34(2):216–224

    Article  Google Scholar 

  6. Gong C, Tao D, Liu W, Maybank SJ, Fang M, Fu K, Yang J (2015) Saliency propagation from simple to difficult. IEEE conference on computer vision and pattern recognition, pp 2531–2539

  7. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Aaron C, Bengio Y (2014) Generative adversarial nets. Advances in neural information processing systems, pp 2672–2680

  8. Guanying H, Qingwu L, Xinnan F (2010) A fast super-resolution algorithm with despeckling for multi-frame sonar images. IEEE International Conference on Information Science and Engineering, pp 3412–3415

  9. Gulrajani I, Ahmed F, Arjovsky M, Dumoulin V, Courville A (2017) Improved training of wasserstein gans. Neural Information Processing Systems, pp 5769–5779

  10. Han W, Chang S, Liu D, Yu M, Witbrock M, Huang TS (2018) Image super-resolution via dual-state recurrent networks. IEEE Conference on Computer Vision and Pattern Recognition, pp 107–112

  11. Haris M, Shakhnarovich G, Ukita N. (2018) Deep back-projection networks for super-resolution, IEEE conference on computer vision and pattern recognition, pp 1664–1673

  12. Harris JL (1964) Diffraction and resolving power. J Opt Soc Am 54(7):931–933

    Article  Google Scholar 

  13. Hou HS, Andrews HC (1987) Cubic splines for image interpolation and digital filtering, IEEE transaction on acoustics. Speech Ans Signal Process 26 (6):508–517

    Google Scholar 

  14. Irani M, Peleg S (1991) Improving resolution by image registration. Graph Models Image Process 53(1):231–239

    Article  Google Scholar 

  15. Kim J, Kwon Lee J, Mu Lee K (2016) Accurate image super-resolution using very deep convolutional networks. IEEE conference on computer vision and pattern recognition, pp 1646–1654

  16. Kim J, Lee JK, Lee KM (2016) Deeply-Recursive Convolutional network for image Super-Resolution IEEE conference on computer vision and pattern recognition

  17. Kingma DP, Ba J (2014) Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980

  18. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7533):436–444

    Article  Google Scholar 

  19. Ledig C, Theis L, Huszar F, Caballero J, Shi W (2017) Photo-Realistic Single image Super-Resolution using a generative adversarial network. IEEE Conference on Computer Vision and Pattern Recognition, pp 105–114

  20. Li X, Orchard MTF (2001) New Edge-Directed interpolation. IEEE Trans Image Process 10(10):1521–1527

    Article  Google Scholar 

  21. Li X, Orchard MT (2001) New Edge-Directed interpolation. IEEE Trans Image Process 10(10):1521–1527

    Article  Google Scholar 

  22. Liu S, Li X (2019) A novel image super-resolution reconstruction algorithm based on improved GANs and gradient penalty. Inter J Intel Comp Cyber 12 (3):400–413

    Article  Google Scholar 

  23. Ma C, Yang CY, Yang X, Yang MH (2017) Learning a no-reference quality metric for single-image super-resolution. Comput Vis Image Underst 158:1–16

    Article  Google Scholar 

  24. Mittal A, Soundararajan R, Bovik AC (2013) Making a “completely blind” image quality analyzer. IEEE Signal Process Lett 20(3):209–212

    Article  Google Scholar 

  25. Park J, Ku B, Jin Y, Ko H (2019) Side scan sonar image super resolution via Region-Selective sparse coding. IEICE Trans Infor and Syst 102(1):210–213

    Article  Google Scholar 

  26. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma Sean, Huang Z, Karpathy A, Khosla A, Bernstein M, Berg AC, Fei-Fei L (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115 (3):211–252

    Article  MathSciNet  Google Scholar 

  27. Schultz RR, Stevenson RL (1994) A bayesian approach to image expansion for improved definition. IEEE Trans Image Process 3(3):233–242

    Article  Google Scholar 

  28. Schultz RR, Stevenson RL (1995) Improved definition video frame enhancement, IEEE international conference on acoustics. Speech and Signal Processing, pp 2169–2172

  29. Schultz RR, Stevenson RL (1996) Extraction of High-Resolution frames from video sequences. IEEE Trans Image Process 5(6):996–1011

    Article  Google Scholar 

  30. Shi W, Caballerob J, Huszar F, Totz J, Aitken AP, Bishop R, Rueckert D, Wang Z (2016) Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. IEEE Conference on Computer Vision and Pattern Recognition, pp 1874–1883

  31. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R (2014) Dropout: a simple way to prevent neural networks from ovefitting. J Machine Learning Research 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  32. Stark H, Oskoui P (1989) High resolution image recovery from Image-Plane arrays, using convex projections, journal of the optical society of america. A Optics and Image Sci 6(11):1715–1726

    Article  Google Scholar 

  33. Sung M, Kim J, Yu SC (2018) Image-based super resolution of underwater sonar images using generative adversarial network. IEEE Region 10 Conference TENCON, pp 0457–0461

  34. Timofte R, De Smet V, Van Gool L. (2014) A+: Adjusted anchored neighborhood regression for fast super-resolution. Asian conference on computer vision, pp 111–126

  35. Valsesia D, Fracastoro G, Magli E (2020) Deep graph-convolutional image denoising. IEEE Trans Image Process 29:8226–8237

    Article  MathSciNet  Google Scholar 

  36. Wang X, Yu K, Wu S, Gu J, Liu Y, Dong C, Qiao Y, Change LC (2018) Esrgan: enhanced super-resolution generative adversarial networks. European Conference on Computer Vision, pp 1–23

  37. Yu J, Xiao CB, Su KN (2006) A method of Gibbs artifact reduction for POCS super-resolution image reconstruction. IEEE International Conference on Signal Processing, pp s262–265

  38. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. European Conference on Computer Vision, pp 286–301

  39. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2018) Residual dense network for image super-resolution. IEEE Conference on Computer Vision and Pattern Recognition, pp 2472–2481

  40. Zhao JX, Liu JJ, Fan DP, Cao Y, Yang JF, Chen MM (2019) EGNEt: Edge guidance network for salient object detection. IEEE International Conference on Computer Vision, pp 8779–8788

Download references

Acknowledgements

The authors would like to thank Prof. Song Hongtao from Harbin Engineering University, Heilongjiang, China, and Ms. Liang Xuecan from Alibaba Group, for their valuable discussions of this paper, and the editor and anonymous reviewers for their constructive comments.

This work is supported by by National Natural Science Foundation of China (61501132), China Postdoctoral Science Foundation (2019M661319), Heilongjiang Postdoctoral Scientic Research Developmental Foundation (LBH-Q17042), and Fundamental Research Funds for the Central Universities (3072020CFQ0602, 3072020CF0604).

Author information

Authors and Affiliations

  1. Harbin Engineering University, Harbin, China

    Pengyang Shen, Liguo Zhang, Minghao Wang & Guisheng Yin

Authors
  1. Pengyang Shen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liguo Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Minghao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guisheng Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liguo Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shen, P., Zhang, L., Wang, M. et al. Deeper super-resolution generative adversarial network with gradient penalty for sonar image enhancement. Multimed Tools Appl 80, 28087–28107 (2021). https://doi.org/10.1007/s11042-021-10888-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10888-y

Keywords

Search

Navigation