Skip to main content
Log in

Enhanced visual perception for underwater images based on multistage generative adversarial network

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Underwater images often suffer from color distortion and low contrast, which dramatically affects the target detection and measurement tasks in the underwater context. In this paper, we present a multistage generative adversarial network for better visual perception of underwater images. Extensive multi-scale context feature learning and high-precision restoration of spatial details are implemented stage by stage. Rich context features are learned based on the encoder and decoder architecture. Spatial details are restored through a pixel restoration module based on original images. Through channel attention module used between multistages, cross-stage feature utilization is realized. More notably, we introduce Gaussian noise into the generator, which enriches the details of images, and the relative discriminator, which promotes the generated image to have more realistic edges and textures. Experimental results demonstrate the superiority of our method over state-of-the-art methods in terms of both quantitative metrics and visual quality. In particular, we applied our method to natural underwater scenes. The results confirm that our method can effectively improve the efficiency of downstream tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Mhala, N.C., Pais, A.R.: A secure visual secret sharing (vss) scheme with cnn-based image enhancement for underwater images. The Visual Computer, 1–15 (2020)

  2. Wu, M., Luo, K., Dang, J., Li, D.: Underwater image restoration using color correction and non-local prior. In: OCEANS 2017-Aberdeen, pp. 1–5 (2017). IEEE

  3. Qiao, N., Di, L.: Underwater image enhancement combining low-dimensional and global features. The Visual Computer, 1–11 (2022)

  4. Pang, Y., Wu, C., Wu, H., Yu, X.: Over-sampling strategy-based class-imbalanced salient object detection and its application in underwater scene. The Visual Computer, 1–16 (2022)

  5. Wu, Y., Zhou, Y., Chen, S., Ma, Y., Li, Q.: Defect inspection for underwater structures based on line-structured light and binocular vision. Appl. Opt. 60(25), 7754–7764 (2021)

    Article  Google Scholar 

  6. Ghani, A.S.A., Isa, N.A.M.: Underwater image quality enhancement through integrated color model with rayleigh distribution. Appl. Soft Comput. 27, 219–230 (2015)

    Article  Google Scholar 

  7. A.R.S.M., S.M.H.: Underwater image enhancement using single scale retinex on a reconfigurable hardware. In: 2015 International Symposium on Ocean Electronics (SYMPOL), pp. 1–5 (2015). https://doi.org/10.1109/SYMPOL.2015.7581166

  8. Jia, Y., Rong, C., Wu, C., Yang, Y.: Research on the decomposition and fusion method for the infrared and visible images based on the guided image filtering and gaussian filter. In: 2017 3rd IEEE International Conference on Computer and Communications (ICCC), pp. 1797–1802 (2017). https://doi.org/10.1109/CompComm.2017.8322849

  9. Deng, X., Zhang, Y., Wang, H., Hu, H.: Robust underwater image enhancement method based on natural light and reflectivity. JOSA A 38(2), 181–191 (2021)

    Article  Google Scholar 

  10. Lin, R., Liu, J., Liu, R., Fan, X.: Global structure-guided learning framework for underwater image enhancement. The Visual Computer, 1–16 (2021)

  11. Huang, D., Wang, Y., Song, W., Sequeira, J., Mavromatis, S.: Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition. In: International Conference on Multimedia Modeling, pp. 453–465 (2018). Springer

  12. Shao, G., Gao, F., Li, T., Zhu, R., Pan, T., Chen, Y.: An adaptive image contrast enhancement algorithm based on retinex. In: 2020 Chinese Automation Congress (CAC), pp. 6294–6299 (2020). IEEE

  13. Fan, T., Li, C., Ma, X., Chen, Z., Zhang, X., Chen, L.: An improved single image defogging method based on retinex. In: 2017 2nd International Conference on Image, Vision and Computing (ICIVC), pp. 410–413 (2017). IEEE

  14. Parihar, A.S., Singh, K.: A study on retinex based method for image enhancement. In: 2018 2nd International Conference on Inventive Systems and Control (ICISC), pp. 619–624 (2018). IEEE

  15. Gunawan, A.A.S., Setiadi, H.: Handling illumination variation in face recognition using multiscale retinex. In: 2016 International Conference on Advanced Computer Science and Information Systems (ICACSIS), pp. 470–475 (2016). https://doi.org/10.1109/ICACSIS.2016.7872757

  16. Chowdhury, D., Das, S.K., Nandy, S., Chakraborty, A., Goswami, R., Chakraborty, A.: An atomic technique for removal of gaussian noise from a noisy gray scale image using lowpass-convoluted gaussian filter. In: 2019 International Conference on Opto-Electronics and Applied Optics (Optronix), pp. 1–6 (2019). https://doi.org/10.1109/OPTRONIX.2019.8862330

  17. Khan, A., Ali, S.S.A., Malik, A.S., Anwer, A., Meriaudeau, F.: Underwater image enhancement by wavelet based fusion. In: 2016 IEEE International Conference on Underwater System Technology: Theory and Applications (USYS), pp. 83–88 (2016). https://doi.org/10.1109/USYS.2016.7893927

  18. Bhatia, N., Kumar Rawat, T.: An improved technique for image contrast enhancement using wavelet transforms. In: 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon), pp. 815–819 (2017). https://doi.org/10.1109/SmartTechCon.2017.8358486

  19. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011). https://doi.org/10.1109/TPAMI.2010.168

    Article  Google Scholar 

  20. Drews Jr, P., do Nascimento, E., Moraes, F., Botelho, S., Campos, M.: Transmission estimation in underwater single images. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 825–830 (2013). https://doi.org/10.1109/ICCVW.2013.113

  21. Ancuti, C.O., Ancuti, C., De Vleeschouwer, C., Garcia, R.: Locally adaptive color correction for underwater image dehazing and matching. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 997–1005 (2017). https://doi.org/10.1109/CVPRW.2017.136

  22. Wang, Y., Liu, H., Chau, L.-P.: Single underwater image restoration using adaptive attenuation-curve prior. IEEE Trans. Circuits Syst. I Regul. Pap. 65(3), 992–1002 (2018). https://doi.org/10.1109/TCSI.2017.2751671

    Article  Google Scholar 

  23. Akkaynak, D., Treibitz, T.: Sea-thru: A method for removing water from underwater images. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1682–1691 (2019). https://doi.org/10.1109/CVPR.2019.00178

  24. Yang, H.-H., Huang, K.-C., Chen, W.-T.: Laffnet: a lightweight adaptive feature fusion network for underwater image enhancement. IET Image Processing (2021)

  25. Panetta, K., Kezebou, L., Oludare, V., Agaian, S.: Comprehensive underwater object tracking benchmark dataset and underwater image enhancement with gan. IEEE J. Ocean. Eng. (2021). https://doi.org/10.1109/JOE.2021.3086907

    Article  Google Scholar 

  26. Guo, Y., Li, H., Zhuang, P.: Underwater image enhancement using a multiscale dense generative adversarial network. IEEE J. Oceanic Eng. 45(3), 862–870 (2020). https://doi.org/10.1109/JOE.2019.2911447

    Article  Google Scholar 

  27. Liu, X., Gao, Z., Chen, B.M.: Mlfcgan: Multilevel feature fusion-based conditional gan for underwater image color correction. IEEE Geosci. Remote Sens. Lett. 17(9), 1488–1492 (2020). https://doi.org/10.1109/LGRS.2019.2950056

    Article  Google Scholar 

  28. Li, C.Y., Cavallaro, A.: Cast-gan: Learning to remove colour cast from underwater images. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1083–1087 (2020). https://doi.org/10.1109/ICIP40778.2020.9191157

  29. Islam, M.J., Xia, Y., Sattar, J.: Fast underwater image enhancement for improved visual perception. IEEE Robot. Autom. Lett. 5(2), 3227–3234 (2020). https://doi.org/10.1109/LRA.2020.2974710

    Article  Google Scholar 

  30. Fabbri, C., Islam, M.J., Sattar, J.: Enhancing underwater imagery using generative adversarial networks. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 7159–7165 (2018). https://doi.org/10.1109/ICRA.2018.8460552

  31. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2015)

  32. Jie, H., Li, S., Gang, S., Albanie, S.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell., p. 99 (2017)

  33. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H., Shao, L.: Multi-stage progressive image restoration. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)

  34. Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C.C., Qiao, Y., Tang, X.: Esrgan: Enhanced super-resolution generative adversarial networks. In: European Conference on Computer Vision (2018)

  35. Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard gan. arXiv preprint arXiv:1807.00734 (2018)

  36. Lai, W.-S., Huang, J.-B., Ahuja, N., Yang, M.-H.: Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans. Pattern Anal. Mach. Intell. 41(11), 2599–2613 (2018)

    Article  Google Scholar 

  37. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision (2016)

  38. Zhao, Y., Wu, R., Dong, H.: Unpaired image-to-image translation using adversarial consistency loss. In: European Conference on Computer Vision, pp. 800–815 (2020). Springer

  39. Fu, Z., Lin, X., Wang, W., Huang, Y., Ding, X.: Underwater image enhancement via learning water type desensitized representations. arXiv preprint arXiv:2102.00676 (2021)

  40. Hanmante, B.P., Ingle, M.: Underwater image restoration based on light absorption. In: 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), pp. 1–4 (2018). https://doi.org/10.1109/ICCUBEA.2018.8697518

  41. Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Oceanic Eng. 41(3), 541–551 (2015)

    Article  Google Scholar 

  42. Chang, Y.-L., Liu, Z.Y., Lee, K.-Y., Hsu, W.: Free-form video inpainting with 3d gated convolution and temporal patchgan. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9066–9075 (2019)

  43. Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)

Download references

Funding

This study was funded by National Key Research and Development Program (2018YFC0406900), the Fundamental Research Funds for the Central Universities (B220201037), Jiangsu Provincial Key Research and Development Program (BE2020649, BE2020092), and National Natural Science Foundation of China (62001156).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yunpeng Ma.

Ethics declarations

Conflict of interest

All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Yu, D., Zhou, Y. et al. Enhanced visual perception for underwater images based on multistage generative adversarial network. Vis Comput 39, 5375–5387 (2023). https://doi.org/10.1007/s00371-022-02665-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02665-1

Keywords

Navigation