Abstract
Underwater images often suffer from color distortion and low contrast, which dramatically affects the target detection and measurement tasks in the underwater context. In this paper, we present a multistage generative adversarial network for better visual perception of underwater images. Extensive multi-scale context feature learning and high-precision restoration of spatial details are implemented stage by stage. Rich context features are learned based on the encoder and decoder architecture. Spatial details are restored through a pixel restoration module based on original images. Through channel attention module used between multistages, cross-stage feature utilization is realized. More notably, we introduce Gaussian noise into the generator, which enriches the details of images, and the relative discriminator, which promotes the generated image to have more realistic edges and textures. Experimental results demonstrate the superiority of our method over state-of-the-art methods in terms of both quantitative metrics and visual quality. In particular, we applied our method to natural underwater scenes. The results confirm that our method can effectively improve the efficiency of downstream tasks.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Mhala, N.C., Pais, A.R.: A secure visual secret sharing (vss) scheme with cnn-based image enhancement for underwater images. The Visual Computer, 1–15 (2020)
Wu, M., Luo, K., Dang, J., Li, D.: Underwater image restoration using color correction and non-local prior. In: OCEANS 2017-Aberdeen, pp. 1–5 (2017). IEEE
Qiao, N., Di, L.: Underwater image enhancement combining low-dimensional and global features. The Visual Computer, 1–11 (2022)
Pang, Y., Wu, C., Wu, H., Yu, X.: Over-sampling strategy-based class-imbalanced salient object detection and its application in underwater scene. The Visual Computer, 1–16 (2022)
Wu, Y., Zhou, Y., Chen, S., Ma, Y., Li, Q.: Defect inspection for underwater structures based on line-structured light and binocular vision. Appl. Opt. 60(25), 7754–7764 (2021)
Ghani, A.S.A., Isa, N.A.M.: Underwater image quality enhancement through integrated color model with rayleigh distribution. Appl. Soft Comput. 27, 219–230 (2015)
A.R.S.M., S.M.H.: Underwater image enhancement using single scale retinex on a reconfigurable hardware. In: 2015 International Symposium on Ocean Electronics (SYMPOL), pp. 1–5 (2015). https://doi.org/10.1109/SYMPOL.2015.7581166
Jia, Y., Rong, C., Wu, C., Yang, Y.: Research on the decomposition and fusion method for the infrared and visible images based on the guided image filtering and gaussian filter. In: 2017 3rd IEEE International Conference on Computer and Communications (ICCC), pp. 1797–1802 (2017). https://doi.org/10.1109/CompComm.2017.8322849
Deng, X., Zhang, Y., Wang, H., Hu, H.: Robust underwater image enhancement method based on natural light and reflectivity. JOSA A 38(2), 181–191 (2021)
Lin, R., Liu, J., Liu, R., Fan, X.: Global structure-guided learning framework for underwater image enhancement. The Visual Computer, 1–16 (2021)
Huang, D., Wang, Y., Song, W., Sequeira, J., Mavromatis, S.: Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition. In: International Conference on Multimedia Modeling, pp. 453–465 (2018). Springer
Shao, G., Gao, F., Li, T., Zhu, R., Pan, T., Chen, Y.: An adaptive image contrast enhancement algorithm based on retinex. In: 2020 Chinese Automation Congress (CAC), pp. 6294–6299 (2020). IEEE
Fan, T., Li, C., Ma, X., Chen, Z., Zhang, X., Chen, L.: An improved single image defogging method based on retinex. In: 2017 2nd International Conference on Image, Vision and Computing (ICIVC), pp. 410–413 (2017). IEEE
Parihar, A.S., Singh, K.: A study on retinex based method for image enhancement. In: 2018 2nd International Conference on Inventive Systems and Control (ICISC), pp. 619–624 (2018). IEEE
Gunawan, A.A.S., Setiadi, H.: Handling illumination variation in face recognition using multiscale retinex. In: 2016 International Conference on Advanced Computer Science and Information Systems (ICACSIS), pp. 470–475 (2016). https://doi.org/10.1109/ICACSIS.2016.7872757
Chowdhury, D., Das, S.K., Nandy, S., Chakraborty, A., Goswami, R., Chakraborty, A.: An atomic technique for removal of gaussian noise from a noisy gray scale image using lowpass-convoluted gaussian filter. In: 2019 International Conference on Opto-Electronics and Applied Optics (Optronix), pp. 1–6 (2019). https://doi.org/10.1109/OPTRONIX.2019.8862330
Khan, A., Ali, S.S.A., Malik, A.S., Anwer, A., Meriaudeau, F.: Underwater image enhancement by wavelet based fusion. In: 2016 IEEE International Conference on Underwater System Technology: Theory and Applications (USYS), pp. 83–88 (2016). https://doi.org/10.1109/USYS.2016.7893927
Bhatia, N., Kumar Rawat, T.: An improved technique for image contrast enhancement using wavelet transforms. In: 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon), pp. 815–819 (2017). https://doi.org/10.1109/SmartTechCon.2017.8358486
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011). https://doi.org/10.1109/TPAMI.2010.168
Drews Jr, P., do Nascimento, E., Moraes, F., Botelho, S., Campos, M.: Transmission estimation in underwater single images. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 825–830 (2013). https://doi.org/10.1109/ICCVW.2013.113
Ancuti, C.O., Ancuti, C., De Vleeschouwer, C., Garcia, R.: Locally adaptive color correction for underwater image dehazing and matching. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 997–1005 (2017). https://doi.org/10.1109/CVPRW.2017.136
Wang, Y., Liu, H., Chau, L.-P.: Single underwater image restoration using adaptive attenuation-curve prior. IEEE Trans. Circuits Syst. I Regul. Pap. 65(3), 992–1002 (2018). https://doi.org/10.1109/TCSI.2017.2751671
Akkaynak, D., Treibitz, T.: Sea-thru: A method for removing water from underwater images. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1682–1691 (2019). https://doi.org/10.1109/CVPR.2019.00178
Yang, H.-H., Huang, K.-C., Chen, W.-T.: Laffnet: a lightweight adaptive feature fusion network for underwater image enhancement. IET Image Processing (2021)
Panetta, K., Kezebou, L., Oludare, V., Agaian, S.: Comprehensive underwater object tracking benchmark dataset and underwater image enhancement with gan. IEEE J. Ocean. Eng. (2021). https://doi.org/10.1109/JOE.2021.3086907
Guo, Y., Li, H., Zhuang, P.: Underwater image enhancement using a multiscale dense generative adversarial network. IEEE J. Oceanic Eng. 45(3), 862–870 (2020). https://doi.org/10.1109/JOE.2019.2911447
Liu, X., Gao, Z., Chen, B.M.: Mlfcgan: Multilevel feature fusion-based conditional gan for underwater image color correction. IEEE Geosci. Remote Sens. Lett. 17(9), 1488–1492 (2020). https://doi.org/10.1109/LGRS.2019.2950056
Li, C.Y., Cavallaro, A.: Cast-gan: Learning to remove colour cast from underwater images. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1083–1087 (2020). https://doi.org/10.1109/ICIP40778.2020.9191157
Islam, M.J., Xia, Y., Sattar, J.: Fast underwater image enhancement for improved visual perception. IEEE Robot. Autom. Lett. 5(2), 3227–3234 (2020). https://doi.org/10.1109/LRA.2020.2974710
Fabbri, C., Islam, M.J., Sattar, J.: Enhancing underwater imagery using generative adversarial networks. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 7159–7165 (2018). https://doi.org/10.1109/ICRA.2018.8460552
Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2015)
Jie, H., Li, S., Gang, S., Albanie, S.: Squeeze-and-excitation networks. IEEE Trans. Pattern Anal. Mach. Intell., p. 99 (2017)
Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.H., Shao, L.: Multi-stage progressive image restoration. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021)
Wang, X., Yu, K., Wu, S., Gu, J., Liu, Y., Dong, C., Loy, C.C., Qiao, Y., Tang, X.: Esrgan: Enhanced super-resolution generative adversarial networks. In: European Conference on Computer Vision (2018)
Jolicoeur-Martineau, A.: The relativistic discriminator: a key element missing from standard gan. arXiv preprint arXiv:1807.00734 (2018)
Lai, W.-S., Huang, J.-B., Ahuja, N., Yang, M.-H.: Fast and accurate image super-resolution with deep laplacian pyramid networks. IEEE Trans. Pattern Anal. Mach. Intell. 41(11), 2599–2613 (2018)
Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: European Conference on Computer Vision (2016)
Zhao, Y., Wu, R., Dong, H.: Unpaired image-to-image translation using adversarial consistency loss. In: European Conference on Computer Vision, pp. 800–815 (2020). Springer
Fu, Z., Lin, X., Wang, W., Huang, Y., Ding, X.: Underwater image enhancement via learning water type desensitized representations. arXiv preprint arXiv:2102.00676 (2021)
Hanmante, B.P., Ingle, M.: Underwater image restoration based on light absorption. In: 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), pp. 1–4 (2018). https://doi.org/10.1109/ICCUBEA.2018.8697518
Panetta, K., Gao, C., Agaian, S.: Human-visual-system-inspired underwater image quality measures. IEEE J. Oceanic Eng. 41(3), 541–551 (2015)
Chang, Y.-L., Liu, Z.Y., Lee, K.-Y., Hsu, W.: Free-form video inpainting with 3d gated convolution and temporal patchgan. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9066–9075 (2019)
Bochkovskiy, A., Wang, C.-Y., Liao, H.-Y.M.: Yolov4: Optimal speed and accuracy of object detection. arXiv preprint arXiv:2004.10934 (2020)
Funding
This study was funded by National Key Research and Development Program (2018YFC0406900), the Fundamental Research Funds for the Central Universities (B220201037), Jiangsu Provincial Key Research and Development Program (BE2020649, BE2020092), and National Natural Science Foundation of China (62001156).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhang, S., Yu, D., Zhou, Y. et al. Enhanced visual perception for underwater images based on multistage generative adversarial network. Vis Comput 39, 5375–5387 (2023). https://doi.org/10.1007/s00371-022-02665-1
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-022-02665-1