Skip to main content
SpringerLink
Log in

Underwater image enhancement based on multiscale fusion generative adversarial network

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The underwater optical imaging environment presents unique challenges due to its complexity. This paper addresses the limitations of existing algorithms in handling underwater images captured in artificial light scenes. We proposed an underwater artificial light optimization algorithm to preprocess images with uneven lighting, mitigating the effects of light distortion. Furthermore, we proposed a novel underwater image enhancement algorithm based the Multiscale Fusion Generative Adversarial Network, named UMSGAN, to address the issues of low contrast and color distortion. UMSGAN uses the generative adversarial network as the underlying framework and first extracts information from the degraded image through three parallel branches separately, and adds residual dense blocks in each branch to learn deeper features. Subsequently, the features extracted from the three branches are fused and the detailed information of the image is recovered by the reconstruction module, named RM. Finally, multiple loss functions are linearly superimposed, and the adversarial network is trained iteratively to obtain the enhanced underwater images. The algorithm is designed to accommodate various underwater scenes, providing both color correction and detail enhancement. We conducted a comprehensive evaluation of the proposed algorithm, considering both qualitative and quantitative aspects. The experimental results demonstrate the effectiveness of our approach on a diverse underwater image dataset. The proposed algorithm exhibits superior performance in terms of enhancing underwater image quality, achieving significant improvements in contrast, color accuracy, and detail preservation. The proposed methodology exhibits promising results, offering potential applications in various domains such as underwater photography, marine exploration, and underwater surveillance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

The datasets analysed during the current study are available in the Github repository, https://github.com/xahidbuffon/funie-gan. The datasets generated during the current study are available from the corresponding author on reasonable request.

References

  1. Yi X, Walia E, Babyn P (2019) Generative adversarial network in medical imaging: a review. Med Image Anal. https://doi.org/10.1016/j.media.2019.101552

    Article  PubMed  Google Scholar 

  2. Khaldi Y, Benzaoui A (2020) A new framework for grayscale ear images recognition using generative adversarial networks under unconstrained conditions. Evol Syst. https://doi.org/10.1007/s12530-020-09346-1

    Article  Google Scholar 

  3. Guo C, Li C, Guo J, Cong R, Fu H, Han P (2019) Hierarchical Features Driven Residual Learning for Depth Map Super-Resolution. In: IEEE transactions on image processing. https://doi.org/10.1109/TIP.2018.2887029

  4. Li Y, Li J, Li Y, Kim H, Serikawa S (2019) Low-light underwater image enhancement for deep-sea tripod. In: IEEE Access. https://doi.org/10.1109/ACCESS.2019.2897691

  5. Hou M, Liu R, Fan X, Luo Z (2018) Joint residual learning for underwater image enhancement. In: 2018 25th IEEE international conference on image processing (ICIP). https://doi.org/10.1109/ICIP.2018.8451209

  6. Zhang S, Wang T, Dong J, Yu H (2017) Underwater image enhancement via extended multi-scale retinex. Neurocomputing. https://doi.org/10.1016/j.neucom.2017.03.029

    Article  PubMed  PubMed Central  Google Scholar 

  7. Fabbri C, Islam MJ, Sattar J (2018) Enhancing underwater imagery using generative adversarial networks. In: Proc. IEEE Int. Conf. Robot. Automat 7159–7165.

  8. Wang Y, Fan H et al (2021) BA-GAN: Block Attention GAN model for Underwater Image Enhancement. In: Proc. ICUS. https://doi.org/10.1109/ICUS52573.2021.9641455

  9. Song W, Jing XJ et al (2021) Underwater image enhancement based on generative adversarial network with preprocessed image penalty. Laser & Optoelectronics Progress. 1006–4125

  10. Estrada DC, Dalgleish FR et al (2022) Underwater LiDAR image enhancement using a GAN based machine learning technique. IEEE Sens J. https://doi.org/10.1109/JSEN.2022.3146133

    Article  Google Scholar 

  11. Islam MJ, Xia Y, Sattar J (2020) Fast underwater image enhancement for improved visual perception. IEEE Robotics and Automation Letters. https://doi.org/10.1109/LRA.2020.2974710

    Article  Google Scholar 

  12. Muthuraman DL, Santhanam SM (2021) Visibility improvement of underwater turbid image using hybrid restoration network with weighted filter. Multidimension Syst Signal Process. https://doi.org/10.1007/s11045-021-00795-8

    Article  Google Scholar 

  13. Farbman Z, Fattal R, Lischinski D et al (2008) Edge-preserving decompositions for multi-scale tone and detail manipulation. Acm Trans Graph 10(1145/1360612):1360666

    Google Scholar 

  14. Li C, Guo C et al (2019) An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2019.2955241

    Article  PubMed  Google Scholar 

  15. Qi Q, Zhang Y et al (2022) Underwater image co-enhancement with correlation feature matching and joint learning. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2021.3074197

    Article  Google Scholar 

  16. Drews JP, Nascimento E et al (2013) Transmission Estimation in Underwater Single Images. In: Proc. ICCVW. https://doi.org/10.1109/ICCVW.2013.113

  17. Hanmante BP, Ingle M (2018) Underwater image restoration based on light absorption. In: Proc. ICCUBEA

  18. Wei S, Yan W et al (2018) A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration. In: Proc. https://doi.org/10.1007/978-3-030-00776-8_62

  19. Liu S, Fan H et al (2022) Adaptive learning attention network for underwater image enhancement. In: IEEE Robot Auto Lett. https://doi.org/10.1109/LRA.2022.3156176

  20. Chen XL, Zhang P, Quan LW et al (2021) Underwater image enhancement based on deep learning and image formation model. arXiv. https://doi.org/10.48550/arXiv.2101.00991

  21. Gonzalez-sabbagh S, Robles-kelly A, Gao S (2022) DGD-cGAN:a Dual Generator for Image Dewatering and Restoration. arXiv. https://doi.org/10.48550/arXiv.2211.10026

  22. Jiang CQ, Xue Z (2019) Application of hybrid residual learning and guided filtering algorithm in image defogging. Opt Precision Eng 2702−2712

  23. Isola P, Zhu JY et al (2016) Image-to-image translation with conditional adversarial networks. IEEE. https://doi.org/10.1109/CVPR.2017.632

    Article  Google Scholar 

  24. Mirza M, Osindero S (2014) Conditional generative adversarial nets. Comput Sci. https://doi.org/10.48550/arXiv.1411.1784

  25. Li C, Wand M (2016) Precomputed real-time texture synthesis with markovian generative adversarial networks. Springer. https://doi.org/10.1007/978-3-319-46487-9_43

    Article  ADS  Google Scholar 

  26. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. Comput Vis Pattern Recognit. https://doi.org/10.48550/arXiv.1409.1556

  27. Zhou Y, Yan KM, Li XF (2022) Underwater image enhancement via physical-feedback adversarial transfer learning. IEEE J Oceanic Eng. https://doi.org/10.1109/JOE.2021.3104055

    Article  Google Scholar 

  28. Sankur LB (2002) Statistical evaluation of image quality measures. J Electron Imaging 10(1117/1):1455011

    Google Scholar 

  29. Zhou W, Bovik AC et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2003.819861

    Article  PubMed  Google Scholar 

  30. Peng YT, Cosman PC (2017) Underwater image restoration based on image blurriness and light absorption. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2017.2663846

    Article  MathSciNet  PubMed  Google Scholar 

  31. Yang M, Sowmya A (2015) An underwater color image quality evaluation metric. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2015.2491020

    Article  MathSciNet  PubMed  Google Scholar 

  32. Ma H, Xin J, Chen, et al (2018) Speckle noise reduction in optical coherence tomography images based on edge-sensitive cGAN. Biomed Opt Express. https://doi.org/10.1364/BOE.9.005129

    Article  PubMed  PubMed Central  Google Scholar 

  33. Jian M, Liu X, Luo H (2021) Underwater image processing and analysis: a review. Signal Process Image Commun. https://doi.org/10.1016/j.image.2020.116088

    Article  Google Scholar 

  34. Yang M, Hu K, Du YX, Wei ZQ et al (2020) Underwater image enhancement based on conditional generative adversarial network. Signal Process. https://doi.org/10.1016/j.image.2019.115723

    Article  Google Scholar 

  35. Langenkmper D, Kevelaer VR, Purser A et al (2020) Gear-induced concept drift in marine images and its effect on deep learning classification. Front Mar Sci. https://doi.org/10.3389/fmars.2020.00506

    Article  Google Scholar 

  36. Xiao J, Zhang S, Yao Y et al (2022) Generative adversarial network with hybrid attention and compromised normalization for multi-scene image conversion. Neural Comput Appl. https://doi.org/10.1007/s00521-021-06841-7

    Article  PubMed  PubMed Central  Google Scholar 

  37. Wang Z, Xuan J, Shi T (2022) A novel semi-supervised generative adversarial network based on the actor-critic algorithm for compound fault recognition. Neural Comput Appl. https://doi.org/10.1007/s00521-022-07011-z

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Automation, Nanjing University of Science and Technology, Nanjing, 210094, Jiangsu, China

    Yating Dai, Jianyu Wang, Hao Wang & Xin He

Authors
  1. Yating Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jianyu Wang.

Ethics declarations

Conflict of interest

We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dai, Y., Wang, J., Wang, H. et al. Underwater image enhancement based on multiscale fusion generative adversarial network. Int. J. Mach. Learn. & Cyber. 15, 1331–1341 (2024). https://doi.org/10.1007/s13042-023-01970-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-023-01970-y

Keywords

Search

Navigation