Skip to main content
SpringerLink
Log in

A natural-based fusion strategy for underwater image enhancement

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Underwater images generally are characterized by color cast and low contrast due to selective absorption and light scattering in water medium. Such degraded images reveal some limitations when used for further analysis. To overcome underwater image degradation, various enhancement techniques are developed. Especially, the fusion-based methods have made remarkable success in this filed. However, there are still some defects in the fusion of input images and weight maps, which cause their results to be unnatural. In this paper, we propose a novel and effective natural-based fusion method for underwater image enhancement that applies several image processing algorithms. First, we design an adaptive underwater image white balance method motivated by our statistical prior to mitigate the impact of color deviation of underwater scenes. We then derive two inputs that represent local detail-improved and global contrast-enhanced versions of the color corrected image. Instead of explicitly estimating weight map, like most existing algorithms, we propose a naturalness-preserving weight map estimation (NP-WME) method, which models the weight map estimation as an optimization problem. Particle swarm optimization (PSO) is used to solve it. Benefiting a proper weighting, the proposed method can achieve a trade-off between detail enhancement and contrast improvement, resulting a natural appearance of the fused image. Through this synthesis, we merge the advantages of different algorithms to obtain the output image. Experimental results show that the proposed method outperforms the several related methods based on quantitative and qualitative evaluations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P et al (2012) Slic superpixels compared to state-of-the-art superpixel methods. IEEE Transactions on Pattern Analysis & Machine Intelligence 34(11):2274– 2282

    Article  Google Scholar 

  2. Akkaynak D, Tali T (2018) A revised underwater image formation model. In: IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 6723–6732

  3. Ancuti C, Ancuti CO, Haber T, Bekaert P (2012) Enhancing underwater images and videos by fusion. IEEE Conf Comput Vis Pattern Recognit, 81–88

  4. Ancuti CO, Ancuti C, Vleeschouwer CD, Bekaert P (2018) Color balance and fusion for underwater image enhancement. IEEE Trans Image Process 27(1):379–393

    Article  MathSciNet  Google Scholar 

  5. Ancuti CO, Ancuti C, Vleeschouwer CD, Sbert M (2019) Color channel compensation (3c): a fundamental pre-processing step for image enhancement. IEEE Trans Image Process 29:2653–2665

    Article  Google Scholar 

  6. Avidan S, Berman D, Treibitz T (2017) Color restoration of underwater images. In: British Machine Vision Conference (BMVC)

  7. Bai L, Zhang W, Pan X, Zhao C (2020) Underwater image enhancement based on global and local equalization of histogram and dual-image multi-scale fusion. IEEE Access 8:128973–128990

    Article  Google Scholar 

  8. Berman D, Levy D, Avidan S, Treibitz T (2018)

  9. Boom BJ, et al. (2014) A research tool for long-term and continuous analysis of fish assemblage in coral-reefs using underwater camera footage. Ecological Informat 23(9):83–97

    Article  Google Scholar 

  10. Buchsbaum G (1980) A spatial processor model for object colour perception. J Frankl Inst 310(1):1–26

    Article  Google Scholar 

  11. Chambah M, Semani D, Renouf A, et al. (2004) Underwater color constancy: Enhancement of automatic live fish recognition. Proceedings of SPIE - The International Society for Optical Engineering

  12. Chiang J-Y, Chen Y-C (2012) Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans Image Process 21(4):1756–1769

    Article  MathSciNet  Google Scholar 

  13. Crete F, Dolmiere T, Ladret P, Nicolas M (2007) The blur effect: Perception and estimation with a new no-reference perceptual blur metric. In: Human Vision and Electronic Imaging XII, p 649201

  14. Daniel B, Kennedy J (2007) Defining a standard for particle swarm optimization. In: 2007 IEEE Swarm Intelligence Symposium

  15. Drews PLJ, Nascimento ER, Botelho SSC, Campos MFM (2016) Underwater depth estimation and image restoration based on single images. IEEE Comput. Graph. Appl 36(2):24–35

    Article  Google Scholar 

  16. Drews PJr, do Nascimento E, Moraes F, Botelho S, Campos M (2013) Transmission estimation in underwater single images. IEEE International Conference on Computer Vision Workshops, 825– 830

  17. Fabbri C, Islam MJ, Sattar J (2018) Enhancing underwater imagery using generative adversarial networks. In: IEEE Int Conf Robot Autom (ICRA), pp 7159–7165

  18. Finlayson GD, Trezzi E (2004) Shades of gray and colour constancy. In: Color and Imaging Conference

  19. Fu X , Fan Z , Ling M , Huang Y, Ding X (2017) Two-step approach for single underwater image enhancement. In: 2017 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS)

  20. Galdran A, Pardo D, Picón A., image A. Alvarez-Gila. (2014) Automatic red-channel underwater restoration. J Vis Commun Image Represent, 26

  21. Guo Y, Li H, Zhuang P (2020) Underwater image enhancement using a multiscale dense generative adversarial network. IEEE J Ocean Eng 45:862–870

    Article  Google Scholar 

  22. Guo F, Liu L, Tang J (2017) Pso-based single image defogging. In: CCF Chinese Conference on Computer Vision

  23. Hassan N, Ullah S, Bhatti N, Mahmood H, Zia M (2021) The retinex based improved underwater image enhancement. Multimed Tools Appl, (80):1839–1857

  24. He JSK, Tang X (2011) Single image haze removal using dark channel prior. IEEE Trans Pattern Anal Mach Intell 33(12):2341–2353

    Article  Google Scholar 

  25. Hou G, Li J, Wang G, Yang H, Huang B, Pan Z (2020) A novel dark channel prior guided variational framework for underwater image restoration. J Vis Commun Image Represent, 66

  26. Hou G, Pan Z, Wang G, Yang H, Duan J (2019) An efficient nonlocal variational method with application to underwater image restoration. Neurocomputing 369:106–121

    Article  Google Scholar 

  27. Iqbal K, Salam RA, Azam O, Talib AZ (2015) Underwater image enhancement using an integrated colour model. IAENG Int J Comput Sci 34(2):239–244

    Google Scholar 

  28. Islam MJ, Xia Y, Sattar J (2020) Fast underwater image enhancement for improved visual perception. IEEE Robot Autom Lett 5(2):3227–3234

    Article  Google Scholar 

  29. Jian M, Qi Q, Dong J, Yin Y, Lam K-M (2018) Integrating qdwd with pattern distinctness and local contrast for underwater saliency detection. J Vis Commun Image Represent 53(3):31–41

    Article  Google Scholar 

  30. Kopf J, et al. (2008) Deep photo: model-based photograph enhancement and viewing. ACM Trans Graph

  31. Land EH (1978) The retinex theory of color vision. Sci Am 237 (6):108–128

    Article  Google Scholar 

  32. Li C, Anwar S, Porikli F (2020) Underwater scene prior inspired deep underwater image and video enhancement. Pattern Recog 98:107038–107049

    Article  Google Scholar 

  33. Li C, Guo J, Cong R, Pang Y, Wang B (2016) Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans Image Process 25(12):5664–5677

    Article  MathSciNet  Google Scholar 

  34. Li C-Y, Guo J-C, Cong R-M, Pang Y-W, Wang B (2016) Underwater image enhancement by dehazing with minimum information loss and histogram distribution prior. IEEE Trans Image Process 25(12):5664–5677

    Article  MathSciNet  Google Scholar 

  35. Li C, Guo J, Guo C (2018) Emerging from water: Underwater image color correction based on weakly supervised color transfer. IEEE Signal Process Lett 25(3):323–327

    Article  Google Scholar 

  36. Li C, Guo C, Ren W, Cong R, Hou J, Kwong S, Tao D (2020) An underwater image enhancement benchmark dataset and beyond. IEEE Trans Image Process

  37. Liang Z, Wang Y, Ding X, Mi Z, Fu X (2020) Single underwater image enhancement by attenuation map guided color correction and detail preserved dehazing. Neurocomputing

  38. Liu R, Fan X, Zhu M, Hou M, Luo Z (2020) Real-world underwater enhancement: Challenges, benchmarks, and solutions under natural light. IEEE Trans Circuits Syst Video Technol

  39. Marques TP, Albu AB (2020) L2uwe: a framework for the efficient enhancement of low-light underwater images using local contrast and multi-scale fusion. CVPR Whorshops, 2286–2295

  40. Mi Z, Li Y, Wang Y, Fu X (2020) Multi-purpose oriented real-world underwater image enhancement. IEEE Access, 112957–112968

  41. Mohd Azmi KZ, Abdul Ghani AS, Md Yusof Z, Ibrahim Z (2019) Natural-based underwater image color enhancement through fusion of swarm-intelligence algorithm. Appli Soft Comput, 85

  42. Narasimhan SG, Nayar SK (2003) Contrast restoration of weather degraded images. IEEE Transactions on Pattern Analysis & Machine Intelligence 25 (6):713–724

    Article  Google Scholar 

  43. Panetta K, Gao C, Agaian S (2016) Human-visual-system-inspired underwater image quality measures. IEEE J Ocean Eng 41(3):541–551

    Article  Google Scholar 

  44. Peng Y-T, Cao K, Cosman PC (2018) Generalization of the dark channel prior for single image restoration. IEEE Trans Image Process, 2856–2868

  45. Peng Y-T, Cosman PC (2017) Underwater image restoration based on image blurriness and light absorption. IEEE Trans Image Process 26(4):1579–1594

    Article  MathSciNet  Google Scholar 

  46. Roznere M, Li AQ (2019) Real-time model-based image color correction for underwater robots. IEEE IROS, 7191–7196

  47. Schechner YY, Averbuch Y (2007) Regularized image recovery in scattering media. IEEE Transactions on Pattern Analysis & Machine Intelligence 29 (9):1655–60

    Article  Google Scholar 

  48. Sethi R, Sreedevi I (2019) Adaptive enhancement of underwater images using multi-objective pso. Multimed Tools Appl 78(22):31823–31845

    Article  Google Scholar 

  49. Sharma G, Wu W, Dalal EN (2005) The ciede2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations. Color Research & Application 30(1):21–30

    Article  Google Scholar 

  50. Shi Y (1998) A modified particle swarm optimizer. In: Proc of IEEE Icec conference

  51. Shi Y, Eberhart RC (1999) Empirical study of particle swarm optimization. In: Proceedings of the 1999 congress on evolutionary computation CEC 99

  52. Shiqi W, Kede M, Hojatollah Y, Zhou W, Weisi L (2015) A patch-structure representation method for quality assessment of contrast changed images. IEEE Signal Processing Letters 22(12):2387–2390

    Article  Google Scholar 

  53. Torres-Méndez LA, Dudek G (2005) Color correction of underwater images for aquatic robot inspection. In: International Workshop on Energy Minimization Methods in Computer Vision & Pattern Recognition, pp 60–73

  54. Wagner B, Nascimento ER, Barbosa WV, Campos MFM (2018) Single-shot underwater image restoration: a visual quality-aware method based on light propagation model. J Vis Commun Image Represent, 55

  55. Wang Y, Liu H, Chau L (2018) Single underwater image restoration using adaptive attenuation-curve prior. IEEE Trans Circuits Syst I Regul Pap 65(3):992–1002

    Article  Google Scholar 

  56. Wang D, Tan D, Liu L (2018) Particle swarm ptimization algoritm: an overview. Appl. Soft Comput 22(2):387–408

    Article  Google Scholar 

  57. Wang S, Zheng J, Hu H-M (2013) Naturalness preserved enhancement algorithm for non-uniform illumination images. IEEE Trans Image Process 22(9):3538–3548

    Article  Google Scholar 

  58. Weijer Jvd, Gevers T, Gijsenij A (2007) Edge-based color constancy. IEEE Trans Image Process 16(9):2207–2214

    Article  MathSciNet  Google Scholar 

  59. Yang M, Hu K, Du Y, Wei Z, Hu J (2019) Underwater image enhancement based on conditional generative adversarial network. Signal Process Image Commun, 81

  60. Yu H, Li X, Lou Q, Lei C, Liu Z (2020) Underwater image enhancement based on dcp and depth transmission map. Multimed Tools Appl 79:20373–20390

    Article  Google Scholar 

  61. Zhang S, Wang T, Dong J, Yu H (2017) Underwater image enhancement via extended multi-scale retinex. Neurocomputing 245:1–9

    Article  Google Scholar 

  62. Zhou J , Liu Z , Zhang W , Zhang D , Zhang W (2020) Underwater image restoration based on secondary guided transmission map. Multim Tools Appl, 7771–7788

Download references

Acknowledgements

The authors sincerely thank the editors and anonymous reviewers for the very helpful and kind comments to assist in improving the presentation of our paper. This work was supported in part by the National Natural Science Foundation of China under Grant 62176037, Grant 62002043, and Grant 61802043, by the Liaoning Revitalization Talents Program under Grant XLYC1908007, by the Foundation of Liaoning Key Research and Development Program under Grant 201801728, by the Dalian Science and Technology Innovation Fund under Grant 2018J12GX037, Grant 2019J11CY001, and Grant 2021JJ12GX028.

Author information

Authors and Affiliations

  1. Information Science and Technology School, Dalian Maritime University, Dalian, 116026, China

    Xiaohong Yan, Guangxin Wang, Guangqi Jiang, Yafei Wang, Zetian Mi & Xianping Fu

  2. Pengcheng Laboratory, Shenzhen, Guangdong, 518055, China

    Xianping Fu

Authors
  1. Xiaohong Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guangxin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guangqi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yafei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zetian Mi
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xianping Fu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Yafei Wang or Xianping Fu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, X., Wang, G., Jiang, G. et al. A natural-based fusion strategy for underwater image enhancement. Multimed Tools Appl 81, 30051–30068 (2022). https://doi.org/10.1007/s11042-022-12267-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12267-7

Keywords

Search

Navigation