Skip to main content
SpringerLink
Log in

An efficient swin transformer-based method for underwater image enhancement

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Due to the complex imaging environment of the ocean, the underwater images obtained by optical vision systems are usually severely degraded. Recently, methods for enhancing underwater images are mostly based on deep learning. However, the intrinsic locality of convolution operation makes it difficult to model long-range dependency efficiently, which may lead to the limited performance of these methods. This paper proposes an efficient method for underwater image enhancement by utilizing Swin Transformer for local feature learning and long-range dependency modeling. The network structure of this method is mainly composed of encoder, decoder and skip connections, in which the encoder and decoder take the Swin Transformer block as the basic unit. Specifically, the encoder is used to learn multi-scale feature representations, and the decoder is utilized to upsample the extracted contextual features progressively. Skip connections are used to fuse multi-scale features from the encoder and decoder. Experimental results demonstrate that the proposed method outperforms state-of-the-art methods on different datasets by up to 1.09\(\sim \)1.64dB (PSNR) and 1.9%\(\sim \)2.3% (SSIM) in objective metrics, and achieves the best visual effect in subjective comparisons, especially in terms of color cast removal and sharpness enhancement.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data Availability

The datasets generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. http://csms.haifa.ac.il/profiles/tTreibitz/datasets/ambient_forwardlooking/index.html

References

  1. Berman D, Levy D, Avidan S, Treibitz T (2021) Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans Pattern Anal Mach Intell 43(8):2822–2837

    Google Scholar 

  2. Bhatti UA, Huang M, Wu D, Zhang Y, Mehmood A, Han H (2019) Recommendation system using feature extraction pattern recognition in clinical care systems. Enterp Inf Syst 13(3):329–351

    Article  Google Scholar 

  3. Bhatti UA et al (2020) Geometric algebra applications in geospatial artificial intelligence and remote sensing image processing. IEEE Access 8:155783–155796

    Article  Google Scholar 

  4. Bhatti UA et al (2021) Advanced color edge detection using clifford algebra in satellite images. IEEE Photonics J 13(2):1–20

    Article  Google Scholar 

  5. Bhatti UA et al (2022) Local similarity-based spatial–spectral fusion hyperspectral image classification with deep CNN and gabor filtering. IEEE Trans Geosci Remote Sens 60:1–15

    Article  Google Scholar 

  6. Cao J et al (2021) Video super-resolution transformer. arXiv:2106.06847

  7. Carion N et al (2020) End-to-end object detection with transformers. In: Eur conf comput vis. Springer, Cham, pp 213–229

  8. Chen C-FR, Fan Q, Panda R (2021) Crossvit: cross-attention multi-scale vision transformer for image classification. In: IEEE int conf comput vis, pp 357–366

  9. Chen D-J, Hsieh H-Y, Liu T-L (2021) Adaptive image transformer for one-shot object detection. In: IEEE conf comput vis pattern recognit, pp 12242–12251

  10. Chen H et al (2021) Pre-trained image processing transformer. In: IEEE conf comput vis pattern recognit, pp 12299–12310

  11. Chen L et al (2021) Perceptual underwater image enhancement with deep learning and physical priors. IEEE Trans Circuits Syst Video Technol 31 (8):3078–3092

    Article  Google Scholar 

  12. Dai Z, Cai B, Lin Y, Chen J (2021) UP-DETR: unsupervised pre-training for object detection with transformers. In: IEEE conf comput vis pattern recognit, pp 1601–1610

  13. Dosovitskiy A et al (2020) An image is worth 16x16 words: transformers for image recognition at scale. In: Int conf learn represent

  14. Drews PJ, Do Nascimento E, Moraes F, Botelho S, Campos M (2013) Transmission estimation in underwater single images. In: IEEE int conf comput vis workshops, pp 825–830

  15. Gao S-B, Zhang M, Zhao Q, Zhang X-S, Li Y-J (2019) Underwater image enhancement using adaptive retinal mechanisms. IEEE Trans Image Process 28(11):5580–5595

    Article  MathSciNet  MATH  Google Scholar 

  16. Guo Y, Li H, Zhuang P (2020) Underwater image enhancement using a multiscale dense generative adversarial network. IEEE J Oceanic Eng 45 (3):862–870

    Article  Google Scholar 

  17. He K, Sun J, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Article  Google Scholar 

  18. Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: Int conf pattern recognit, pp 2366–2369

  19. Hu J, Jiang Q, Cong R, Gao W, Shao F (2021) Two-branch deep neural network for underwater image enhancement in HSV color space. IEEE Signal Process Lett 28:2152–2156

    Article  Google Scholar 

  20. Islam MJ, Xia Y, Sattar J (2020) Fast underwater image enhancement for improved visual perception. IEEE Rob Autom Lett 5(2):3227–3234

    Article  Google Scholar 

  21. Jaffe JS (1990) Computer modeling and the design of optimal underwater imaging systems. IEEE J Oceanic Eng 15(2):101–111

    Article  Google Scholar 

  22. Johnson J, Alahi A, Li F (2016) Perceptual losses for real-time style transfer and super-resolution. In: Eur conf comput vis. Springer, Cham, pp 694–711

  23. Korhonen J, You J (2012) Peak signal-to-noise ratio revisited: is simple beautiful?. In: 2012 Fourth international workshop on quality of multimedia experience (QoMEx), pp 37–38

  24. Lanchantin J, Wang T, Ordonez V, Qi Y (2021) General multi-label image classification with transformers. In: IEEE conf comput vis pattern recognit, pp 16473–16483

  25. Li Y, Chen R (2021) UDA-Net: densely attention network for underwater image enhancement. IET Image Proc 15(3):774–785

    Article  Google Scholar 

  26. Li H, Zhuang P (2021) Dewaternet: a fusion adversarial real underwater image enhancement network. Signal Process Image Commun, vol 95(116248)

  27. Li C, Anwar S, Porikli F (2020) Underwater scene prior inspired deep underwater image and enhancement. Video Pattern recognit, vol 98(107038)

  28. Li C et al (2020) An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Process 29:4376–4389

    Article  MATH  Google Scholar 

  29. Li C, Anwar S, Hou J, Cong R, Guo C, Ren W (2021) Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Trans Image Process 30:4985–5000

    Article  Google Scholar 

  30. Liang J, Cao J, Sun G, Zhang K, Van Gool L, Timofte R (2021) Swinir: image restoration using swin transformer. In: IEEE int conf comput vis, pp 1833–1844

  31. Liu P, Wang G, Qi H, Zhang C, Zheng H, Yu Z (2019) Underwater image enhancement with a deep residual framework. IEEE Access 7:94614–94629

    Article  Google Scholar 

  32. Liu R, Fan X, Zhu M, Hou M, Luo Z (2020) Real-world underwater enhancement: challenges, benchmarks, and solutions under natural light. IEEE Trans Circuits Syst Video Technol 30(12):4861–4875

    Article  Google Scholar 

  33. Liu X, Gao Z, Chen BM (2020) MLFCGAN: multilevel feature fusion-based conditional GAN for underwater image color correction. IEEE Geosci Remote Sens Lett 17(9):1488–1492

    Article  Google Scholar 

  34. Liu Z et al (2021) Swin transformer: hierarchical vision transformer using shifted windows. In: IEEE int conf comput vis, pp 10012–10022

  35. Mao J et al (2021) Voxel transformer for 3D object detection. In: IEEE int conf comput vis, pp 3144–3153

  36. Misra I, Girdhar R, Joulin A (2021) An end-to-end transformer model for 3D object detection. In: IEEE int conf comput vis, pp 2906–2917

  37. Moghimi MK, Mohanna F (2021) Real-time underwater image enhancement: a systematic review. J Real-Time Image Process 18(5):1509–1525

    Article  Google Scholar 

  38. Panetta K, Gao C, Agaian S (2016) Human-visual-system-inspired underwater image quality measures. IEEE J Oceanic Eng 41(3):541–551

    Article  Google Scholar 

  39. Peng L, Zhu C, Bian L (2021) U-shape transformer for underwater image enhancement. arXiv:2111.11843

  40. Sajid U, Chen X, Sajid H, Kim T, Wang G (2021) Audio-visual transformer based crowd counting. In: IEEE int conf comput vis workshops, pp 2249–2259

  41. Singhai J, Rawat P (2007) Image enhancement method for underwater, ground and satellite images using brightness preserving histogram equalization with maximum entropy. In: IEEE int conf comput intell multimed appl, pp 507–512

  42. Song W et al (2018) A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration. In: Pacific rim conf multimed. Springer, Cham, pp 678–688

  43. Song W, Wang Y, Huang D, Liotta A, Perra C (2020) Enhancement of underwater images with statistical model of background light and optimization of transmission map. IEEE Trans Broadcast 66(1):153–169

    Article  Google Scholar 

  44. Srinivas A, Lin T-Y, Parmar N, Shlens J, Abbeel P, Vaswani A (2021) Bottleneck transformers for visual recognition. In: IEEE conf comput vis pattern recognit, pp 16519–16529

  45. Touvron H et al (2021) Training data-efficient image transformers & distillation through attention. In: Int conf mach learn, pp 10347–10357

  46. Vaswani A et al (2017) Attention is all you need. In: Adv neural inf process syst, pp 5998–6008

  47. Wang J et al (2020) CA-GAN: class-condition attention GAN for underwater image enhancement. IEEE Access 8:130719–130728

    Article  Google Scholar 

  48. Wang Y et al (2021) End-to-end video instance segmentation with transformers. In: IEEE conf comput vis pattern recognit, pp 8737–8746

  49. Yan K et al (2022) Medium transmission map matters for learning to restore real-world underwater images. Appl Sci 12(11):5420

    Article  Google Scholar 

  50. Yang M, Sowmya A (2015) An underwater color image quality evaluation metric. IEEE Trans Image Process 24(12):6062–6071

    Article  MathSciNet  MATH  Google Scholar 

  51. Yang M, Hu J, Li C, Rohde G, Du Y, Hu K (2019) An in-depth survey of underwater image enhancement and restoration. IEEE Access 7:123638–123657

    Article  Google Scholar 

  52. Yu H, Li X, Lou Q, Lei C, Liu Z (2020) Underwater image enhancement based on DCP and depth transmission map. Multimed Tools Appl 79:20373–20390

    Article  Google Scholar 

  53. Zhang Z, Lu X, Cao G, Yang Y, Jiao L, Liu F (2021) ViT-YOLO: transformer-based YOLO for object detection. In: IEEE int conf comput vis, pp 2799–2808

  54. Zhang W et al (2021) Enhancing underwater image via color correction and bi-interval contrast enhancement. Signal Process Image Commun, vol 90(116030)

  55. Zhao H, Gallo O, Frosio I, Kautz J (2017) Loss functions for image restoration with neural networks. IEEE Trans Comput Imaging 3(1):47–57

    Article  Google Scholar 

  56. Zheng S et al (2021) Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers. In: IEEE conf comput vis pattern recognit, pp 6877–6886

  57. Zhuang P, Ding X (2020) Correction to: underwater image enhancement using an edge-preserving filtering Retinex algorithm. Multimed Tools Appl 79 (25):17257–17277

    Article  Google Scholar 

  58. Zhuang P, Li C, Wu J (2021) Bayesian retinex underwater image enhancement. Eng Appl Artif Intell, vol 101(104171)

Download references

Acknowledgements

This work was supported by the Key Research and Development Project of Hainan Province (No. ZDYF2019024).

Author information

Authors and Affiliations

  1. School of Information and Communication Engineering, Hainan University, No. 58 Renmin Avenue, Haikou, 570228, China

    Rong Wang, Yonghui Zhang & Jian Zhang

Authors
  1. Rong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yonghui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yonghui Zhang.

Ethics declarations

Conflict of Interests

The authors declare no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, R., Zhang, Y. & Zhang, J. An efficient swin transformer-based method for underwater image enhancement. Multimed Tools Appl 82, 18691–18708 (2023). https://doi.org/10.1007/s11042-022-14228-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14228-6

Keywords

Search

Navigation