Skip to main content
SpringerLink
Log in

Gated aggregation network for cloud detection in remote sensing image

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Cloud detection is one of the important tasks in remote sensing image preprocessing, and this paper uses RGB preview images of remote sensing images to extract cloud regions efficiently. The preview images make the detection of cloud regions more challenging due to the lack of resolution and spectral information. The existing remote sensing image cloud detection methods, and feature fusion process, due to unreasonable feature fusion strategy, so that the encoded features cannot be fully utilized, which may introduce noise information, and ultimately lead to the problems of false detection and missing detection. To address these problems, this work designs a gated aggregation network (GANet) for remote sensing image cloud detection. GANet has a novel encoder–decoder architecture, a gated feature aggregation module (GFAM), and a pyramidal attention pooling module (PAPM). GFAM bridges the gap between high resolution with spatial details and low-resolution features with high-level semantics, fully selectively fusing semantic and spatial features to alleviate the semantic divide problem when fusing multi-level features. PAPM extracts multi-scale global contextual features without loss of resolution. The method is validated on three datasets: the publicly available 38-Cloud and SPARCS datasets and the self-built Landsat-8 cloud detection dataset with higher spatial resolution. The experimental results show that the proposed method achieves competitive performance under different evaluation metrics. Codes and datasets can be found at https://github.com/HaiLei-Fly/GANet/.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

The data and the code of this study are open source at https://github.com/HaiLei-Fly/GANet and https://github.com/HaiLei-Fly/CHLandsat8.

References

  1. Li, Z., Zhang, Y., Shao, J., et al.: Remote sensing of atmospheric particulate mass of dry PM2.5 near the ground: method validation using ground-based measurements. Remote Sens. Environ. 173, 59–68 (2016). https://doi.org/10.1016/j.rse.2015.11.019

    Article  Google Scholar 

  2. Stöckli, R., Bojanowski, J.S., John, V.O., et al.: Cloud detection with historical geostationary satellite sensors for climate applications. Remote Sens. 11(9), 1052 (2019). https://doi.org/10.3390/rs11091052

    Article  Google Scholar 

  3. Magney, T.S., Vierling, L.A., Eitel, J.U.H., et al.: Response of high frequency photochemical reflectance index (PRI) measurements to environmental conditions in wheat. Remote Sens. Environ. 173, 84–97 (2016). https://doi.org/10.1016/j.rse.2015.11.013

    Article  Google Scholar 

  4. Yang, F., Guo, J., Tan, H., et al.: Automated extraction of urban water bodies from ZY-3 multi-spectral imagery. Water 9(2), 144 (2017). https://doi.org/10.3390/w9020144

    Article  Google Scholar 

  5. Thus, G.: Cloud removal in satellite images using spatiotemporal generative networks. Suppl. Mater. (2020). https://doi.org/10.1109/WACV45572.2020.9093564

    Article  Google Scholar 

  6. Rossow, W.B., Garder, L.C.: Cloud detection using satellite measurements of infrared and visible radiances for ISCCP. J. Clim. 6(12), 2341–2369 (1993). https://doi.org/10.1175/1520-0442(1993)006%3c2341:CDUSMO%3e2.0.CO;2

    Article  Google Scholar 

  7. Gesell, G.: An algorithm for snow and ice detection using AVHRR data An extension to the APOLLO software package. Int. J. Remote Sens. 10(4–5), 897–905 (1989). https://doi.org/10.1080/01431168908903929

    Article  Google Scholar 

  8. Ackerman, S.A., Strabala, K.I., Menzel, W.P., et al.: Discriminating clear sky from clouds with MODIS. J. Geophys. Res. Atmos. 103(D24), 32141–32157 (1998). https://doi.org/10.1029/1998JD200032

    Article  Google Scholar 

  9. Zhan, Y., Wang, J., Shi, J., et al.: Distinguishing cloud and snow in satellite images via deep convolutional network. IEEE Geosci. Remote Sens. Lett. 14(10), 1785–1789 (2017). https://doi.org/10.1109/LGRS.2017.2735801

    Article  Google Scholar 

  10. Zhu, Z., Wang, S., Woodcock, C.E.: Improvement and expansion of the fmask algorithm: cloud, cloud shadow, and snow detection for landsats 4–7, 8, and sentinel 2 images. Remote Sens. Environ. 159, 269–277 (2015). https://doi.org/10.1016/j.rse.2014.12.014

    Article  Google Scholar 

  11. Zhong, B., Chen, W., Wu, S., et al.: A cloud detection method based on relationship between objects of cloud and cloud-shadow for Chinese moderate to high resolution satellite imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(11), 4898–4908 (2017). https://doi.org/10.1109/JSTARS.2017.2734912

    Article  Google Scholar 

  12. Yang, J., Guo, J., Yue, H., et al.: CDnet: CNN-based cloud detection for remote sensing imagery. IEEE Trans. Geosci. Remote Sens. 57(8), 6195–6211 (2019). https://doi.org/10.1109/TGRS.2019.2904868

    Article  Google Scholar 

  13. Guo, J., Yang, J., Yue, H., et al.: CDnetV2: CNN-based cloud detection for remote sensing imagery with cloud-snow coexistence. IEEE Trans. Geosci. Remote Sens. 59(1), 700–713 (2020). https://doi.org/10.1109/TGRS.2020.2991398

    Article  Google Scholar 

  14. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)

    Article  Google Scholar 

  15. Yang, G.R., Murray, J.D., Wang, X.J.: A dendritic disinhibitory circuit mechanism for pathway-specific gating. Nat. Commun. 7(1), 1–14 (2016)

    Article  Google Scholar 

  16. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 3431–3440 (2015).

  17. Zeng, X., Yang, J., Deng, X. Cloud segmentation of remote sensing images on Landsat-8 by deep learning. In: Proceedings of the 2nd International Conference on Big Data Research.174–177 (2018). https://doi.org/10.1145/3291801.3291839

  18. Ronneberger, O., Fischer, P., Brox, T. U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5–9, 2015, Proceedings, Part III 18. Springer International Publishing, 234–241 (2015).

  19. Badrinarayanan, V., Kendall, A., Cipolla, R.: Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017). https://doi.org/10.1109/TPAMI.2016.2644615

    Article  Google Scholar 

  20. Xie, F., Shi, M., Shi, Z., et al.: Multilevel cloud detection in remote sensing images based on deep learning. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10(8), 3631–3640 (2017). https://doi.org/10.1109/JSTARS.2017.2686488

    Article  Google Scholar 

  21. Mohajerani, S., Krammer, T. A., Saeedi, P.: Cloud detection algorithm for remote sensing images using fully convolutional neural networks. arXiv preprint arXiv:1810.05782, (2018). https://doi.org/10.48550/arXiv.1810.05782

  22. Hu, K., Zhang, D., Xia, M.: CDUNet: cloud detection UNet for remote sensing imagery. Remote Sens. 13(22), 4533 (2021). https://doi.org/10.3390/rs13224533

    Article  Google Scholar 

  23. Mohajerani, S., Saeedi, P.: Cloud-Net: An end-to-end cloud detection algorithm for Landsat 8 imagery. In: IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 1029–1032 (2019). https://doi.org/10.1109/IGARSS.2019.8898776

  24. Mohajerani, S., Saeedi, P.: Cloud and cloud shadow segmentation for remote sensing imagery via filtered jaccard loss function and parametric augmentation. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 14, 4254–4266 (2021). https://doi.org/10.1109/JSTARS.2021.3070786

    Article  Google Scholar 

  25. Lu, C., Xia, M., Lin, H.: Multi-scale strip pooling feature aggregation network for cloud and cloud shadow segmentation. Neural Comput. Appl., 1–14 (2022).

  26. Pu, W., Wang, Z., Liu, D., et al.: Optical remote sensing image cloud detection with self-attention and spatial pyramid pooling fusion. Remote Sens. 14(17), 4312 (2022). https://doi.org/10.3390/rs14174312

    Article  Google Scholar 

  27. Peng, L., Chen, X., Chen, J., et al.: Understanding the role of receptive field of convolutional neural network for cloud detection in landsat 8 OLI imagery. IEEE Trans. Geosci. Remote Sens. 60, 1–17 (2022). https://doi.org/10.1109/TGRS.2022.3150083

    Article  Google Scholar 

  28. Zeiler, M. D., Fergus, R.: Visualizing and understanding convolutional networks. In: Computer Vision–ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6–12, 2014, Proceedings, Part I 13. Springer International Publishing, 818–833 (2014).

  29. Farabet, C., Couprie, C., Najman, L., et al.: Learning hierarchical features for scene labeling. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1915–1929 (2012). https://doi.org/10.1109/TPAMI.2012.231

    Article  Google Scholar 

  30. Zhao, H., Shi, J., Qi, X., et al.: Pyramid scene parsing network. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2881–2890 (2017).

  31. Chen, L.C., Papandreou, G., Kokkinos, I., et al.: Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 834–848 (2017). https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  32. Wang, W., Shi, Z.: An all-scale feature fusion network with boundary point prediction for cloud detection. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2021). https://doi.org/10.1109/LGRS.2021.3110869

    Article  Google Scholar 

  33. Guo, H., Bai, H., Qin, W.: ClouDet: a dilated separable CNN-based cloud detection framework for remote sensing imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 14, 9743–9755 (2021). https://doi.org/10.1109/JSTARS.2021.3114171

    Article  Google Scholar 

  34. Zhao, C., Zhang, X., Luo, H., et al.: Detail-aware multiscale context fusion network for cloud detection. IEEE Geosci. Remote Sens. Lett. 19, 1–5 (2022). https://doi.org/10.1109/LGRS.2022.3207426

    Article  Google Scholar 

  35. Amirul Islam, M., Rochan, M., Bruce, N. D. B. et al.: Gated feedback refinement network for dense image labeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 3751–3759 (2017).

  36. Zhang, L., Dai, J., Lu, H., et al.: A bi-directional message passing model for salient object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 1741–1750 (2018).

  37. Guo, Y., Cao, X., Liu, B., et al.: Cloud detection for satellite imagery using attention-based U-Net convolutional neural network. Symmetry 12(6), 1056 (2020). https://doi.org/10.3390/sym12061056

    Article  Google Scholar 

  38. Xia, M., Wang, T., Zhang, Y., et al.: Cloud/shadow segmentation based on global attention feature fusion residual network for remote sensing imagery. Int. J. Remote Sens. 42(6), 2022–2045 (2021). https://doi.org/10.1080/01431161.2020.1849852

    Article  Google Scholar 

  39. De Boer, P.T., Kroese, D.P., Mannor, S., et al.: A tutorial on the cross-entropy method. Ann. Oper. Res. 134, 19–67 (2005)

    Article  MathSciNet  Google Scholar 

  40. He, K., Zhang, X., Ren, S., et al.: Identity mappings in deep residual networks. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14. Springer International Publishing, 630–645 (2016).

  41. Hughes, M.J., Hayes, D.J.: Automated detection of cloud and cloud shadow in single-date Landsat imagery using neural networks and spatial post-processing. Remote Sens. 6(6), 4907–4926 (2014). https://doi.org/10.3390/rs6064907

    Article  Google Scholar 

  42. Achanta, R., Hemami, S., Estrada, F., et al.: Frequency-tuned salient region detection. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, 1597–1604 (2009). https://doi.org/10.1109/CVPR.2009.5206596

  43. Wang, W., Lai, Q., Fu, H., et al.: Salient object detection in the deep learning era: an in-depth survey. IEEE Trans. Pattern Anal. Mach. Intell. 44(6), 3239–3259 (2021). https://doi.org/10.1109/TPAMI.2021.3051099

    Article  Google Scholar 

  44. Fu, K., Fan, D.P., Ji, G.P., et al.: Siamese network for RGB-D salient object detection and beyond. IEEE Trans. Pattern Anal. Mach. Intell. 44(9), 5541–5559 (2021). https://doi.org/10.1109/TPAMI.2021.3073689

    Article  Google Scholar 

  45. Margolin, R., Zelnik-Manor, L., Tal, A.: How to evaluate foreground maps?. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 248–255 (2014).

  46. Fan, D. P., Cheng, M. M., Liu, Y., et al.: Structure-measure: A new way to evaluate foreground maps. In: Proceedings of the IEEE international conference on computer vision. 4548–4557 (2017).

Download references

Funding

This work is supported in part by the National Natural Science Foundation of China (62241307), the Scientific and Technological Project of Gansu Province (22YF7FA166), the Scientific and Technological Project of Lanzhou City (2022-RC-60), and the University Innovation Fund Project of Gansu Province (2021A-027).

Author information

Authors and Affiliations

  1. College of Electrical and Information Engineering, Lanzhou University of Technology, Lanzhou, 730050, China

    Xianjun Du & Hailei Wu

Authors
  1. Xianjun Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hailei Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xianjun Du.

Ethics declarations

Conflict of interest

No potential conflict of interest was reported by the authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Du, X., Wu, H. Gated aggregation network for cloud detection in remote sensing image. Vis Comput 40, 2517–2536 (2024). https://doi.org/10.1007/s00371-023-02934-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-023-02934-7

Keywords

Search

Navigation