Skip to main content
SpringerLink
Log in

A real-world underwater image enhancement method based on multi-color space and two-stage adaptive fusion

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Due to the influence of light scattering, absorption and noise, the underwater environment presents a range of challenges for image processing tasks, such as color cast, low contrast and poor readability. It is truly regrettable that most algorithms primarily rely on synthetic datasets or few real-world river underwater images for evaluation. To address these challenges, we build a river underwater image enhancement dataset and propose an enhancement approach that leverages multi-color space, prior knowledge of underwater imaging and two-stage adaptive fusion. First, the shallow preprocessing module utilizes deformable convolution to better extract cross information. Next, the parallel RGB and HSV enhancement channels (with attention mechanism) can effectively highlight image characteristics in different colorspaces. Considering the underwater imaging prior, our method reduces the probability of artifacts. Finally, we merge the different frequency features in the first stage. The HSV and RGB components are fused adaptively in the second stage. Extensive experimental results verify the effectiveness of our algorithm in enhancing real-world underwater images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Availability of data and materials

Not applicable.

References

  1. Jaffe, J.S.: Computer modeling and the design of optimal underwater imaging systems. IEEE J. Ocean. Eng. 15(2), 101–111 (1990)

    Article  ADS  Google Scholar 

  2. Hu, K., Weng, C., Zhang, Y., Jin, J., Xia, Q.: An overview of underwater vision enhancement: from traditional methods to recent deep learning. J. Mar. Sci. Eng. 10(2), 241 (2022)

    Article  Google Scholar 

  3. Jian, M., Liu, X., Luo, H., Lu, X., Yu, H., Dong, J.: Underwater image processing and analysis: a review. Signal Process. Image Commun. 91, 116088 (2021)

    Article  Google Scholar 

  4. Anwar, S., Li, C.: Diving deeper into underwater image enhancement: a survey. Signal Process. Image Commun. 89, 115978 (2020)

    Article  Google Scholar 

  5. Rathi, D., Jain, S., Indu, S.: Underwater fish species classification using convolutional neural network and deep learning. In: 2017 Ninth International Conference on Advances in Pattern Recognition (ICAPR), pp. 1–6. IEEE (2017)

  6. Jian, M., Qi, Q., Dong, J., Yin, Y., Lam, K.-M.: Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection. J. Vis. Commun. Image Represent. 53, 31–41 (2018)

    Article  Google Scholar 

  7. Yeh, C.-H., Lin, C.-H., Kang, L.-W., Huang, C.-H., Lin, M.-H., Chang, C.-Y., Wang, C.-C.: Lightweight deep neural network for joint learning of underwater object detection and color conversion. IEEE Trans. Neural Netw. Learn. Syst. 33(11), 6129–6143 (2021)

    Article  Google Scholar 

  8. Ancuti, C.O., Ancuti, C., De Vleeschouwer, C., Bekaert, P.: Color balance and fusion for underwater image enhancement. IEEE Trans. Image Process. 27(1), 379–393 (2017)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  9. Hummel, R.: Image enhancement by histogram transformation. Comput. Graph. Image Process. 6(2), 184–195 (1977)

    Article  Google Scholar 

  10. Ancuti, C., Ancuti, C.O., Haber, T., Bekaert, P.: Enhancing underwater images and videos by fusion. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 81–88. IEEE (2012)

  11. Drews, P., Nascimento, E., Moraes, F., Botelho, S., Campos, M.: Transmission estimation in underwater single images. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 825–830 (2013)

  12. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    PubMed  Google Scholar 

  13. Perez, J., Attanasio, A.C., Nechyporenko, N., Sanz, P.J.: A deep learning approach for underwater image enhancement. In: Biomedical Applications Based on Natural and Artificial Computing: International Work-Conference on the Interplay Between Natural and Artificial Computation, IWINAC 2017, Corunna, Spain, June 19–23, 2017, Proceedings, Part II, pp. 183–192. Springer (2017)

  14. Wang, Y., Zhang, J., Cao, Y., Wang, Z.: A deep CNN method for underwater image enhancement. In: 2017 IEEE International Conference on Image Processing (ICIP), pp. 1382–1386. IEEE (2017)

  15. Anwar, S., Li, C., Porikli, F.: Deep underwater image enhancement. arXiv preprint arXiv:1807.03528 (2018)

  16. Li, C., Guo, C., Ren, W., Cong, R., Hou, J., Kwong, S., Tao, D.: An underwater image enhancement benchmark dataset and beyond. IEEE Trans. Image Process. 29, 4376–4389 (2019)

    Article  ADS  Google Scholar 

  17. Naik, A., Swarnakar, A., Mittal, K.: Shallow-uwnet: Compressed model for underwater image enhancement (student abstract). In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, pp. 15853–15854 (2021)

  18. Wang, Y., Guo, J., Gao, H., Yue, H.: Uiec\(\hat{}\) 2-net: CNN-based underwater image enhancement using two color space. Signal Process. Image Commun. 96, 116250 (2021)

    Article  Google Scholar 

  19. Wu, S., Luo, T., Jiang, G., Yu, M., Xu, H., Zhu, Z., Song, Y.: A two-stage underwater enhancement network based on structure decomposition and characteristics of underwater imaging. IEEE J. Ocean. Eng. 46(4), 1213–1227 (2021)

    Article  ADS  Google Scholar 

  20. Li, C., Anwar, S., Hou, J., Cong, R., Guo, C., Ren, W.: Underwater image enhancement via medium transmission-guided multi-color space embedding. IEEE Trans. Image Process. 30, 4985–5000 (2021)

    Article  ADS  PubMed  Google Scholar 

  21. Liu, S., Fan, H., Lin, S., Wang, Q., Ding, N., Tang, Y.: Adaptive learning attention network for underwater image enhancement. IEEE Robot. Autom. Lett. 7(2), 5326–5333 (2022)

    Article  Google Scholar 

  22. Li, J., Skinner, K.A., Eustice, R.M., Johnson-Roberson, M.: Watergan: unsupervised generative network to enable real-time color correction of monocular underwater images. IEEE Robot. Autom. Lett. 3(1), 387–394 (2017)

    Google Scholar 

  23. Fabbri, C., Islam, M.J., Sattar, J.: Enhancing underwater imagery using generative adversarial networks. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 7159–7165. IEEE (2018)

  24. Guo, Y., Li, H., Zhuang, P.: Underwater image enhancement using a multiscale dense generative adversarial network. IEEE J. Ocean. Eng. 45(3), 862–870 (2019)

    Article  Google Scholar 

  25. Li, H., Li, J., Wang, W.: A fusion adversarial underwater image enhancement network with a public test dataset. arXiv preprint arXiv:1906.06819 (2019)

  26. Islam, M.J., Xia, Y., Sattar, J.: Fast underwater image enhancement for improved visual perception. IEEE Robot. Autom. Lett. 5(2), 3227–3234 (2020)

    Article  Google Scholar 

  27. Hitam, M.S., Awalludin, E.A., Yussof, W.N.J.H.W., Bachok, Z.: Mixture contrast limited adaptive histogram equalization for underwater image enhancement. In: 2013 International Conference on Computer Applications Technology (ICCAT), pp. 1–5. IEEE (2013)

  28. Wang, Y., Ding, X., Wang, R., Zhang, J., Fu, X.: Fusion-based underwater image enhancement by wavelet decomposition. In: 2017 IEEE International Conference on Industrial Technology (ICIT), pp. 1013–1018. IEEE (2017)

  29. Muniraj, M., Dhandapani, V.: Underwater image enhancement by color correction and color constancy via retinex for detail preserving. Comput. Electr. Eng. 100, 107909 (2022)

    Article  Google Scholar 

  30. Peng, Y.-T., Cosman, P.C.: Underwater image restoration based on image blurriness and light absorption. IEEE Trans. Image Process. 26(4), 1579–1594 (2017)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  31. Zhang, M., Peng, J.: Underwater image restoration based on a new underwater image formation model. IEEE Access 6, 58634–58644 (2018)

    Article  Google Scholar 

  32. Liu, Y., Rong, S., Cao, X., Li, T., He, B.: Underwater single image dehazing using the color space dimensionality reduction prior. IEEE Access 8, 91116–91128 (2020)

    Article  Google Scholar 

  33. Berman, D., Levy, D., Avidan, S., Treibitz, T.: Underwater single image color restoration using haze-lines and a new quantitative dataset. IEEE Trans. Pattern Anal. Mach. Intell. 43(8), 2822–2837 (2020)

    Google Scholar 

  34. Hou, M., Liu, R., Fan, X., Luo, Z.: Joint residual learning for underwater image enhancement. In: 2018 25th IEEE International Conference on Image Processing (ICIP), pp. 4043–4047. IEEE (2018)

  35. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Article  MathSciNet  Google Scholar 

  36. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2223–2232 (2017)

  37. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223. PMLR (2017)

  38. Mathieu, M., Couprie, C., LeCun, Y.: Deep multi-scale video prediction beyond mean square error. arXiv preprint arXiv:1511.05440 (2015)

  39. Zong, X., Chen, Z., Wang, D.: Local-cyclegan: a general end-to-end network for visual enhancement in complex deep-water environment. Appl. Intell. 51, 1947–1958 (2021)

    Article  Google Scholar 

  40. Mirza, M., Osindero, S.: Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784 (2014)

  41. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convolutional networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 764–773 (2017)

  42. Hou, Q., Zhou, D., Feng, J.: Coordinate attention for efficient mobile network design. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13713–13722 (2021)

  43. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  44. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision—ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part II 14, pp. 694–711. Springer (2016)

  45. Peng, Y.-T., Cao, K., Cosman, P.C.: Generalization of the dark channel prior for single image restoration. IEEE Trans. Image Process. 27(6), 2856–2868 (2018)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  46. Drews, P.L., Nascimento, E.R., Botelho, S.S., Campos, M.F.M.: Underwater depth estimation and image restoration based on single images. IEEE Comput. Graph. Appl. 36(2), 24–35 (2016)

    Article  PubMed  Google Scholar 

  47. Li, C.-Y., Guo, J.-C., Cong, R.-M., Pang, Y.-W., Wang, B.: Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans. Image Process. 25(12), 5664–5677 (2016)

    Article  ADS  MathSciNet  Google Scholar 

  48. Zuiderveld, K.: Contrast limited adaptive histogram equalization. Graphics Gems, pp. 474–485 (1994)

  49. Li, C., Anwar, S., Porikli, F.: Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recogn. 98, 107038 (2020)

    Article  Google Scholar 

  50. Yang, M., Sowmya, A.: An underwater color image quality evaluation metric. IEEE Trans. Image Process. 24(12), 6062–6071 (2015)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  51. Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Ocean. Eng. 41(3), 541–551 (2015)

Download references

Funding

This work was supported by the Fundamental Research Funds for the Central Universities of China (No. N2216010), the ‘Jie Bang Gua Shuai’ Science and Technology Major Project of Liaoning Province in 2022 (No.2022JH1/10400025) and the National Key Research and Development Program of China (No. 2018YFB1702000).

Author information

Author notes

  1. Weimin Lei and Wei Zhang have contributed equally to this work.

Authors and Affiliations

  1. Computer Science and Engineering, Northeastern University, No.195, Chuangxin Road, Hunnan District, Shenyang, 110169, Liaoning Province, China

    Kai Ji, Weimin Lei & Wei Zhang

Authors
  1. Kai Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weimin Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

KJ wrote the manuscript; WL and WZ provided suggestions for the manuscript. All authors reviewed the manuscript.

Corresponding author

Correspondence to Weimin Lei.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ji, K., Lei, W. & Zhang, W. A real-world underwater image enhancement method based on multi-color space and two-stage adaptive fusion. SIViP 18, 2135–2149 (2024). https://doi.org/10.1007/s11760-023-02864-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02864-w

Keywords

Search

Navigation