Skip to main content
Springer Nature Link
Log in

USIR-Net: sand-dust image restoration based on unsupervised learning

  • RESEARCH
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In sand-dust weather, the influence of sand-dust particles on imaging equipment often results in images with color deviation, blurring, and low contrast, among other issues. These problems making many traditional image restoration methods unable to accurately estimate the semantic information of the images and consequently resulting in poor restoration of clear images. Most current image restoration methods in the field of deep learning are based on supervised learning, which requires pairing and labeling a large amount of data, and the possibility of manual annotation errors. In light of this, we propose an unsupervised sand-dust image restoration network. The overall model adopts an improved CycleGAN to fit unpaired sand-dust images. Firstly, multiscale skip connections in the multiscale cascaded attention module are used to enhance the feature fusion effect after downsampling. Secondly, multi-head convolutional attention with multiple input concatenations is employed, with each head using different kernel sizes to improve the ability to restore detail information. Finally, the adaptive decoder-encoder module is used to achieve adaptive fitting of the model and output the restored image. According to the experiments conducted on the dataset, the qualitative and quantitative indicators of USIR-Net are superior to the selected comparison algorithms, furthermore, in additional experiments conducted on haze removal and underwater image enhancement, we have demonstrated the wide applicability of our model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Irani, B., Wang, J., Chen, W.: A localizability constraint-based path planning method for autonomous vehicles. IEEE Trans. Intell. Transp. Syst. 20(7), 2593–2604 (2019). https://doi.org/10.1109/TITS.2018.2868377

    Article  Google Scholar 

  2. Tan, R.T.: Visibility in bad weather from a single image. In: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Anchorage, AK, USA, pp. 24–26 (2008)

  3. Voronin, V., Tokareva, S., Semenishchev, E., Agaian, S.: Thermal image enhancement algorithm using local and global logarithmic transform histogram matching with spatial equalization. In: 2018 IEEE Southwest Symposium on Image Analysis and Interpretation (SSIAI), Las Vegas, NV, USA, pp. 5–8 (2018).https://doi.org/10.1109/SSIAI.2018.8470344

  4. Jindal, K., Gupta, K., Jain, M., Maheshwari, M.: Bio-medical image enhancement based on spatial domain technique. In: 2014 International Conference on Advances in Engineering & Technology Research (ICAETR - 2014), Unnao, India, pp. 1–5 (2014). https://doi.org/10.1109/ICAETR.2014.7012932.

  5. Ma, J.: Based on the fourier transform and the wavelet transformation of the digital image processing. In: 2012 International Conference on Computer Science and Information Processing (CSIP), Xi'an, China, pp. 1232–1234 (2012). https://doi.org/10.1109/CSIP.2012.6309081

  6. Wen, H., Qi, W., Shuang, L.: Medical X-ray image enhancement based on wavelet domain homomorphic filtering and CLAHE. In: 2016 International Conference on Robots & Intelligent System (ICRIS), ZhangJiaJie, China, pp. 249–254 (2016). https://doi.org/10.1109/ICRIS.2016.50

  7. Cantor, A.: Optics of the atmosphere–Scattering by molecules and particles. IEEE J. Quantum Electron. 14(9), 698–699 (1978). https://doi.org/10.1109/JQE.1978.1069864

    Article  Google Scholar 

  8. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011). https://doi.org/10.1109/TPAMI.2010.168

    Article  Google Scholar 

  9. Fattal, R.: Dehazing using color-lines. ACM Trans Graph 34(1), 1–14 (2014)

    Article  Google Scholar 

  10. Engin, D., Genc, A., Ekenel, H.K.: Cycle-Dehaze: Enhanced CycleGAN for Single Image Dehazing. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA, pp. 938–9388 (2018). https://doi.org/10.1109/CVPRW.2018.00127

  11. Wang, J., Xu, Y., Chen, W.: Multi-patch and feature fusion network for single image Dehazing. In: 2020 International Conference on Intelligent Computing, Automation and Systems (ICICAS), Chongqing, China, pp. 282–285 (2020). https://doi.org/10.1109/ICICAS51530.2020.00064

  12. Chiang, J.Y., Chen, Y.-C.: Underwater image enhancement by wavelength compensation and dehazing. IEEE Trans. Image Process. 21(4), 1756–1769 (2012). https://doi.org/10.1109/TIP.2011.2179666

    Article  MathSciNet  Google Scholar 

  13. Wang, N., Zhou, Y., Han, F., Zhu, H., Yao, J.: ‘UWGAN: underwater GAN for real-world underwater color restoration and dehazing (2019). arXiv:1912.10269, doi: https://doi.org/10.48550/arXiv.1912.10269

  14. Deng, S., Wei, M., Wang, J., Liang, L., Xie, H., Wang, M.: DRD-Net: Detail-recovery image deraining via context aggregation networks (2019). arXiv:1908.10267, doi: https://doi.org/10.48550/arXiv.1908.10267

  15. Shin, Y.-G., Sagong, M.-C., Yeo, Y.-J., Ko, S.-J.: Adversarial context aggregation network for low-light image enhancement. In: 2018 Digital Image Computing: Techniques and Applications (DICTA), Canberra, ACT, Australia, pp. 1–5 (2018).https://doi.org/10.1109/DICTA.2018.8615848

  16. Gasparini, F., Schettini, R.: Color correction for digital photographs. In: 12th International Conference on Image Analysis and Processing, 2003. Proceedings., Mantova, Italy, pp. 646–651 (2003). https://doi.org/10.1109/ICIAP.2003.1234123

  17. Jang, W., Son, K., Kim, J., Kang, B.: Auto white balance system using adaptive color samples for mobile devices. In: APCCAS 2008—2008 IEEE Asia Pacific Conference on Circuits and Systems, Macao, China, pp. 1462–1465 (2008). https://doi.org/10.1109/APCCAS.2008.4746307

  18. Jiang, X., Jin, L., He, L., et al.: Research of the color remote sensing images fusing method based on Retinex and edge extraction[J]. Acta Photonica Sin. 36(3), 565–567 (2007)

    Google Scholar 

  19. Liu, H., Li, C., Wan, Y.Q. et al.: Dust images enhancement algorithm based on color transfer[C]. In: Proceedings of the CCF Chinese Conference on Computer Vision, Tianjin, China, pp. 168–179 (2017)

  20. Gao, G., Lai, H., Jia, Z., Liu, Y., Wang, Y.: Sand-dust image restoration based on reversing the blue channel prior. IEEE Photonics J. 12(2), 1–16 (2020). https://doi.org/10.1109/JPHOT.2020.2975833

    Article  Google Scholar 

  21. Borkar, K., Mukherjee, S.: Single image dehazing by approximating and eliminating the additional airlight component. Neurocomputing 400, 294–308 (2020)

    Article  Google Scholar 

  22. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: DehazeNet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016). https://doi.org/10.1109/TIP.2016.2598681

    Article  MathSciNet  Google Scholar 

  23. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: AOD-Net: All-in-One Dehazing Network. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 4780–4788 (2017). https://doi.org/10.1109/ICCV.2017.511

  24. Qin, X., Wang, Z., Bai, Y. et al.: FFA-Net: feature fusion attention network for single image Dehazing[C]. National Conference on Artificial Intelligence. Association for the Advancement of Artificial Intelligence (AAAI), 34(07), 11908–11915 (2020).

  25. Singh, A., Bhave, A., Prasad, D.K.: Single images dehazing for a variety of haze scenarios using back projected pyramid network[J]. Computer Vision – ECCV 2020 Workshops, 166–181 (2020)

  26. Das, S.D., Dutta, S.: Fast deep multi-patch hierarchical network for nonhomogeneous image Dehazing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Seattle, WA, USA, pp. 1994–2001 (2020). https://doi.org/10.1109/CVPRW50498.2020.00249

  27. Yu, Y., Liu, H., Fu, M., Chen, J., Wang, X., Wang, K.: A two-branch neural network for non-homogeneous Dehazing via ensemble learning. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Nashville, TN, USA, pp. 193–202 (2021). https://doi.org/10.1109/CVPRW53098.2021.00028

  28. Wu, H. et al.: Contrastive learning for compact single image Dehazing. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Nashville, TN, USA, pp. 10546–10555 (2021). https://doi.org/10.1109/CVPR46437.2021.01041

  29. Yang, X., Xu, Z., Luo, J.: Towards perceptual image dehazing by physics-based disentanglement and adversarial training. In: AAAI (2018)

  30. Zhu, J.-Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 2242–2251 (2017). https://doi.org/10.1109/ICCV.2017.244

  31. Anvari, Z., Athitsos, V.: Dehaze-GLCGAN: unpaired single image dehazing via adversarial training. Arxive 2008.06632 (2020)

  32. Chaitanya, B.S.N.V., Mukherjee, S.: Single image dehazing using improved cycleGAN. J. Vis. Commun. Image Represent. 74, 103014 (2021)

    Article  Google Scholar 

  33. Rahman, M.M., Marculescu, R.: Medical image segmentation via cascaded attention decoding. In: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, pp. 6211–6220 (2023). https://doi.org/10.1109/WACV56688.2023.00616

  34. Dai, J. et al.: Deformable convolutional networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), Venice, Italy, pp. 764–773 (2017). https://doi.org/10.1109/ICCV.2017.89

  35. Georgescu, M.-I. et al.: Multimodal multi-head convolutional attention with various kernel sizes for medical image super-resolution. In: 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), Waikoloa, HI, USA, pp. 2194–2204 (2023). https://doi.org/10.1109/WACV56688.2023.00223

  36. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90

  37. Hautière, N., Tarel, J.-P., Aubert, D., Dumont, É.: Blind contrast enhancement assessment by gradient ratioing at visible edges. Image Anal. Stereol. J. 27(2), 87–95 (2008). https://doi.org/10.5566/ias.v27.p87-95

    Article  MathSciNet  Google Scholar 

  38. Mittal, A., Soundararajan, R., Bovik, A.C.: Making a “completely blind” image quality analyzer. IEEE Signal Process. Lett. 20(3), 209–212 (2013). https://doi.org/10.1109/LSP.2012.2227726

    Article  Google Scholar 

  39. Yang, M., Sowmya, A.: An underwater color image quality evaluation metric. IEEE Trans. Image Process. 24(12), 6062–6071 (2015). https://doi.org/10.1109/TIP.2015.2491020

    Article  MathSciNet  Google Scholar 

  40. Zotin, A.G.: Fast algorithm of image enhancement based on multi-scale retinex[J]. Int. J. Reason. Based Intell. Syst. 12(2), 106 (2020)

    Google Scholar 

  41. Huang, D., Wang, Y., Song, W. et al.: Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition[C]. In: International Conference on Multimedia Modeling. Springer, Cham, pp:453–465 (2018)

  42. Park, T.H., Eom, I.K.: Sand-dust image enhancement using successive color balance with coincident chromatic histogram. IEEE Access 9, 19749–19760 (2021). https://doi.org/10.1109/ACCESS.2021.3054899

    Article  Google Scholar 

  43. Ding, B., Chen, H., Xu, L., Zhang, R.: Restoration of single sand-dust image based on style transformation and unsupervised adversarial learning. IEEE Access 10, 90092–90100 (2022). https://doi.org/10.1109/ACCESS.2022.3200163

    Article  Google Scholar 

  44. Yan-ru, F.: Joint image dehazing and transmittance estimation of dual visual attention network[J]. Opt. Precis. Eng. 29(04), 854–863 (2021)

    Article  Google Scholar 

  45. Singh, A., Bhave, A., Prasad, D.K.: Single image dehazing for a variety of haze scenarios using back projected pyramid network [C]. In: Computer Vision–ECCV 2020 Workshops, pp. 166–181 (2020)

  46. Yu, Y.K., Liu, H., Fu, M.H. et al.: A two-branch neural network for non-homogeneous dehazing via ensemble learning. In: 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops(CVPRW). IEEE, Nashville, TN, USA, pp. 193–202 (2021)

  47. Islam, M.J., Xia, Y., Sattar, J.: Fast underwater image enhancement for improved visual perception. IEEE Robot. Autom. Lett. 5(2), 3227–3234 (2020)

    Article  Google Scholar 

  48. Park, T., Efros, A.A., Zhang, R., Zhu, J-Y.: Contrastive learning for unpaired image-to-image translation. In ECCV, pp. 319–345. Springer (2020)

  49. Guo, Z., Guo, D., Gu, Z., Zheng, H., Zheng, B. Wang, G.: Unsupervised underwater image clearness via transformer, OCEANS 2022—Chennai, Chennai, India, pp. 1–4 (2022).https://doi.org/10.1109/OCEANSChennai45887.2022.9775361

Download references

Acknowledgements

This work was supported in part by the Natural Science Foundation of Gansu Province [grant no. 21JR7RA300], in part by the Open Project of Dunhuang Cultural Relics Protection Research Center in Gansu Province [grant no. GDW2021YB15].

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Lanzhou Jiaotong University, Lanzhou, 730070, Gansu, China

    Yuan Ding & Kaijun Wu

Authors
  1. Yuan Ding
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Kaijun Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Yuan Ding: Project administration, Supervision, Conceptualization, Code, Writing—review and editing. Kaijun Wu: Funding acquisition, Methodology, Investigation, Software, Writing—original draft.

Corresponding author

Correspondence to Yuan Ding.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, Y., Wu, K. USIR-Net: sand-dust image restoration based on unsupervised learning. Machine Vision and Applications 35, 45 (2024). https://doi.org/10.1007/s00138-024-01528-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-024-01528-0

Keywords