Skip to main content
SpringerLink
Log in

Multi-scale strip pooling feature aggregation network for cloud and cloud shadow segmentation

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Cloud and cloud shadow detection is a crucial issue in remote sensing image processing. The backgrounds of clouds and cloud shadows are mostly complex in actual remote sensing images. Traditional methods are easily affected by ground object interference, noise interference and other factors, and problems such as missing detection and false detection are prone to occur in the process of cloud detection. In addition, due to insufficient edge information extraction capabilities, traditional methods have very rough segmentation results for cloud and cloud shadow boundaries. In order to improve the accuracy of cloud and cloud shadow detection, a Multi-scale Strip Pooling Feature Aggregation Network is proposed. This method uses the residual network as the backbone to extract different levels of semantic information. And, in order to improve the multi-scale information extraction ability of the network, an Improved Pyramid Pooling module is introduced to mine deep multi-scale semantic information. Then, the Mutual Fusion module is used to guide the fusion of different levels of information. Finally, in view of the problem of rough segmentation boundaries in traditional methods, the Strip Boundary Refinement module is used to repair the boundary information of clouds and cloud shadows. The experimental results conducted on the datasets collected by Landsat-8, Sentinel-2 and a public dataset HRC_WHU show that this method is superior to the existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Zhang Y, Rossow WB, Lacis AA, Oinas V, Mishchenko MI (2004) Calculation of radiative fluxes from the surface to top of atmosphere based on isccp and other global data sets: refinements of the radiative transfer model and the input data. J Geophys Res Atmosp 109:D19105

    Article  Google Scholar 

  2. Shen SS, Descour MR (2000) Algorithms for multispectral, hyperspectral, and ultraspectral imagery vi. Algorithms Multisp Hyperspectr Ultraspectr Imag 4:4049

    Google Scholar 

  3. Liu X, Jianmin X, Bingyu D (2005) A bi-channel dynamic thershold algorithm used in automatically identifying clouds on gms-5 imagery. J Appl Meteorl Sci 16(4):134–444

    Google Scholar 

  4. Ma F, Zhang Q, Guo N, Zhang J (2007) The study of cloud detection with multi-channel data of satellite. Chin J Atmosp Sci Chin Ed 31(1):119

    Google Scholar 

  5. Cao Q, Zheng H, Li X (2007) A method for detecting cloud in satellite remote sensing image based on texture. Acta Aeronaut Astronaut Sin 28(3):661

    Google Scholar 

  6. Molnar G, Coakley JA Jr (1985) Retrieval of cloud cover from satellite imagery data: a statistical approach. J Geophys Res Atmosp 90(D7):12960–12970

    Article  Google Scholar 

  7. Liu Z, Li Y, Huang F (2007) Cloud detection of modis satellite images based on dynamical cluster. Remote Sens Inf 7:33–35

    Google Scholar 

  8. Corneliu Octavian Dumitru and Mihai Datcu (2013) Information content of very high resolution sar images: study of feature extraction and imaging parameters. IEEE Trans Geosci Remote Sens 51(8):4591–4610

    Article  Google Scholar 

  9. Geng J, Fan J, Wang H, Ma X, Li B, Chen F (2015) High-resolution sar image classification via deep convolutional autoencoders. IEEE Geosci Remote Sens Lett 12(11):2351–2355

    Article  Google Scholar 

  10. Liu M, Yan W, Zhao W, Zhang Q, Li M, Liao G (2013) Dempster-shafer fusion of multiple sparse representation and statistical property for sar target configuration recognition. IEEE Geosci Remote Sens Lett 11(6):1106–1110

    Article  Google Scholar 

  11. Tian B, Shaikh MA, Azimi-Sadjadi MR, Haar THV, Reinke DL (1999) A study of cloud classification with neural networks using spectral and textural features. IEEE Trans Neural Netw 10(1):138–151

    Article  Google Scholar 

  12. Gómez-Chova L, Amorós J, Camps-Valls G, Martin JD, Calpe J, Alonso L, Guanter L, Fortea JC, Moreno J (2005) Cloud detection for chris/proba hyperspectral images. In: Remote sensing of clouds and the atmosphere X, vol 5979. International Society for Optics and Photonics, pp 59791Q

  13. Gómez-Chova L, Camps-Valls G, Amorós-López J, Guanter L, Alonso L, Calpe J, Moreno J et al (2006) New cloud detection algorithm for multispectral and hyperspectral images: application to envisat/meris and proba/chris sensors. In: IEEE international geoscience and remote sensing symposium. IGARSS, pp 2757–2760

  14. Gao J, Wang K, Tian X, Chen J (2018) A bp-nn based cloud detection method for fy-4 remote sensing images. J Infrared Millim Waves 37:477–485

    Google Scholar 

  15. Zhang B, Yadong H, Hong J (2021) Cloud detection of remote sensing images based on h-svm with multi-feature fusion. J Atmosp Environ Opt 16(1):58

    Google Scholar 

  16. Xia M, Wang K, Song W, Chen C, Li Y et al (2020) Non-intrusive load disaggregation based on composite deep long short-term memory network. Expert Syst Appl 160:113669

    Article  Google Scholar 

  17. Xia M, Cui Y, Zhang Y, Yiming X, Liu J, Yiqing X (2021) Dau-net: a novel water areas segmentation structure for remote sensing image. Int J Remote Sens 42(7):2594–2621

    Article  Google Scholar 

  18. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst 25:1097–1105

    Google Scholar 

  19. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556

  20. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  21. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  22. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

  23. Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  24. Lafferty J, McCallum A, Pereira FCN (2001) Probabilistic models for segmenting and labeling sequence data, conditional random fields. In: Proceedings of the eighteenth international conference on machine learning, pp 282–289

  25. Zhao H, iShi J, Qi X, Wang X, Jia J (2017) Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2881–2890

  26. Chen B, Xia M, Huang J (2021) Mfanet: a multi-level feature aggregation network for semantic segmentation of land cover. Remote Sens 13(4):731

    Article  Google Scholar 

  27. Zhang L, Sheng Z, Li Y, Sun Q, Zhao Y, Feng D (2019) Image object detection and semantic segmentation based on convolutional neural network. Neural Computi Appl 32:1949–1958

    Article  Google Scholar 

  28. Zhang Y, Li X, Lin M, Chiu B, Zhao M (2020) Deep-recursive residual network for image semantic segmentation. Neural Comput Appl 32(16):12935–12947

    Article  Google Scholar 

  29. Zheng X, Chen T (2021) High spatial resolution remote sensing image segmentation based on the multiclassification model and the binary classification model. Neural Comput Appl. https://doi.org/10.1007/s00521-020-05561-8

    Article  Google Scholar 

  30. Xia M, Wang T, Zhang Y, Liu J, Yiqing X (2021) Cloud/shadow segmentation based on global attention feature fusion residual network for remote sensing imagery. Int J Remote Sens 42(6):2022–2045

    Article  Google Scholar 

  31. Mohajerani S, Saeedi P (2019) Cloud-net: an end-to-end cloud detection algorithm for landsat 8 imagery. In: IGARSS 2019-2019 IEEE international geoscience and remote sensing symposium. IEEE, pp 1029–1032

  32. Zhan Y, Wang J, Shi J, Cheng G, Yao L, Sun W (2017) Distinguishing cloud and snow in satellite images via deep convolutional network. IEEE Geosci Remote Sens Lett 14(10):1785–1789

    Article  Google Scholar 

  33. Li J, Zhao P, Fang W, Song S (2020) Cloud detection of multi-angle remote sensing image based on deep learning. J Atmosp Environ Opt 15(05):380

    Google Scholar 

  34. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  35. Wang X, Girshick R, Gupta A, He K (2018) Non-local neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7794–7803

  36. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. arXiv:1706.03762

  37. Hou Q, Zhang L, Cheng M, Feng J (2020) Strip pooling: rethinking spatial pooling for scene parsing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4003–4012

  38. Hanchao L, Pengfei X, Jie A, Lingxue W (2018) Pyramid attention network for semantic segmentation. arXiv:1805.10180

  39. Yu C, Gao C, Wang J, Yu G, Shen C, Sang N (2020) Bisenet v2: bilateral network with guided aggregation for real-time semantic segmentation. arXiv:2004.02147

  40. Li Z, Shen H, Cheng Q, Liu Y, You S, He Z (2019) Deep learning based cloud detection for medium and high resolution remote sensing images of different sensors. ISPRS J Photogramm Remote Sens 150:197–212

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of PR China (42075130).

Author information

Authors and Affiliations

  1. Jiangsu Key Laboratory of Big Data Analysis Technology, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Chen Lu

  2. Collaborative Innovation Center on Atmospheric Environment and Equipment Technology, Nanjing University of Information Science and Technology, Nanjing, 210044, China

    Min Xia

  3. College of Information Science and Technology, Nanjing Forestry University, Nanjing, 210037, China

    Haifeng Lin

Authors
  1. Chen Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Min Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Haifeng Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Min Xia.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lu, C., Xia, M. & Lin, H. Multi-scale strip pooling feature aggregation network for cloud and cloud shadow segmentation. Neural Comput & Applic 34, 6149–6162 (2022). https://doi.org/10.1007/s00521-021-06802-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-021-06802-0

Keywords

Search

Navigation