Skip to main content
SpringerLink
Log in

Dual-stream shadow detection network: biologically inspired shadow detection for remote sensing images

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Deep learning has achieved state-of-the-art results in various image classification and image segmentation tasks. However, due to the lack of well-labeled datasets, the insufficiency of deep feature extraction, and the complexity of the distribution of shadows on remote sensing images, popular deep neural networks still fall short on satisfactory shadow detection from remote sensing images. Inspired by the brain's mechanism for processing visual signals, this paper proposes a new Dual-stream Shadow Detection Network (DSSDN) that is specifically designed for detecting shadows on remote sensing images. In DSSDN, the pooling stream extracts high-level features by merging multiple atrous pooling feature maps after the encoder, while the residual stream maintains low-level features and carries out the interaction of dual-stream features. This network is also featured with three new sub-modules. We manually labeled 1724 remote sensing images with shadows to form a new dataset for training and testing of DSSDN. In the quantitative contrast experiment on this dataset, DSSDN reaches the lowest Balanced Error Rate (BER) at 6.6% across all compared models and networks. In the qualitative analysis, the detected shadows of DSSDN also show best contours and details in comparison with results from other approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Mostafa Y, Abdelhafiz A (2017) Accurate shadow detection from high-resolution satellite images. IEEE Geosci Remote Sens Lett 14(4):494–498. https://doi.org/10.1109/LGRS.2017.2650996

    Article  Google Scholar 

  2. Hu J, Song Y, Jin T, Lu B, Zhu G, Zhou Z (2015) Shadow effect mitigation in indication of moving human behind wall via MIMO TWIR. IEEE Geosci Remote Sens Lett 12(3):453–457. https://doi.org/10.1109/LGRS.2014.2345777

    Article  Google Scholar 

  3. Sirmacek B, Unsalan C (2009) Damaged building detection in aerial images using shadow information. In: International conference on recent advances in space technologies (RAST '09)

  4. Dare PM (2005) Shadow analysis in high-resolution satellite imagery of urban areas. Photogramm Eng Remote Sens 71:169–177

    Article  Google Scholar 

  5. Huang J, Zhang X, Xin Q, Sun Y, Zhang P (2019) Automatic building extraction from high-resolution aerial images and LiDAR data using gated residual refinement network. ISPRS J Photogramm Remote Sens 151:91–105

    Article  Google Scholar 

  6. Liasis G, Stavrou S (2016) Satellite images analysis for shadow detection and building height estimation. ISPRS J Photogramm Remote Sens 119:437–450

    Article  Google Scholar 

  7. Tsai VJD (2006) A comparative study on shadow compensation of color aerial images in invariant color models. IEEE Trans Geosci Remote Sens 44(6):1661–1671

    Article  Google Scholar 

  8. Yamazaki F, Liu W, Takasaki M (2010) Characteristics of shadow and removal of its effects for remote sensing imagery. In: IEEE geoscience and remote sensing symposium

  9. Makarau A, Richter R, Muller R et al (2011) Adaptive shadow detection using a blackbody radiator model. IEEE Trans Geosci Remote Sens 49(6):2049–2059

    Article  Google Scholar 

  10. Elbakary MI, Iftekharuddin KM (2014) Shadow detection of man-made buildings in high-resolution panchromatic satellite images. IEEE Trans Geosci Remote Sens 52(9):5374–5386

    Article  Google Scholar 

  11. Liasis G, Stavrou S (2016) Satellite images analysis for shadow detection and building height estimation. ISPRS J Photogramm Remote Sens 119:437–450

    Article  Google Scholar 

  12. Tong X, Lin X, Feng T et al (2013) Use of shadows for detection of earthquake-induced collapsed buildings in high-resolution satellite imagery. ISPRS J Photogramm Remote Sens 79:53–67

    Article  Google Scholar 

  13. Movia A, Beinat A, Crosilla F (2016) Shadow detection and removal in RGB VHR images for land use unsupervised classification. ISPRS J Photogramm Remote Sens 119:485–495

    Article  Google Scholar 

  14. Chai D, Newsam S, Zhang HK, Qiu Y, Huang J (2019) Cloud and cloud shadow detection in Landsat imagery based on deep convolutional neural networks. Remote Sens Environ 225:307–316

    Article  Google Scholar 

  15. Morales G, Arteaga D, Samuel G et al (2018) Shadow detection in high-resolution multispectral satellite imagery using generative adversarial networks. In: IEEE international conference on electronics, electrical engineering and computing

  16. Kussul N, Lavreniuk M, Skakun S, Shelestov A (2017) Deep learning classification of land cover and crop types using remote sensing data. IEEE Geosci Remote Sens Lett 14(5):778–782. https://doi.org/10.1109/LGRS.2017.2681128

    Article  Google Scholar 

  17. Qiu C, Mou L, Schmitt M, Zhu XX (2020) Fusing multiseasonal sentinel-2 imagery for urban land cover classification with multibranch residual convolutional neural networks. IEEE Geosci Remote Sens Lett 17(10):1787–1791. https://doi.org/10.1109/LGRS.2019.2953497

    Article  Google Scholar 

  18. Yan Z et al (2018) Cloud and cloud shadow detection using multilevel feature fused segmentation network. IEEE Geosci Remote Sens Lett 15(10):1600–1604. https://doi.org/10.1109/LGRS.2018.2846802

    Article  Google Scholar 

  19. Wang T, Hu X, Wang Q, Heng P-A, Fu C-W (2020) Instance shadow detection. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 1877–1886, https://doi.org/10.1109/CVPR42600.2020.00195

  20. Hu X, Fu C-W, Zhu L, Qin J, Heng P-A (2020) Direction-aware spatial context features for shadow detection and removal. IEEE Trans Pattern Anal Mach Intell 42(11):2795–2808. https://doi.org/10.1109/TPAMI.2019.2919616

    Article  Google Scholar 

  21. Inoue N, Yamasaki T (2020) Learning from synthetic shadows for shadow detection and removal. IEEE Trans Circ Syst Video Technol 31:4187–4197. https://doi.org/10.1109/TCSVT.2020.3047977

    Article  Google Scholar 

  22. Chen Z, Zhu L, Wan L, Wang S, Feng W, Heng P-A (2020) A multi-task mean teacher for semi-supervised shadow detection. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), pp 5610–5619. https://doi.org/10.1109/CVPR42600.2020.00565

  23. Cun X, Pun CM, Shi C (2020) Towards ghost-free shadow removal via dual hierarchical aggregation network and shadow matting GAN. In: AAAI conference on artificial intelligence, pp 10680–10687

  24. Ding B, Long C, Zhang L, Xiao C (2019) ARGAN: attentive recurrent generative adversarial network for shadow detection and removal. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 10212–10221. https://doi.org/10.1109/ICCV.2019.01031

  25. Hou L, Vicente TFY, Hoai M, Samaras D (2021) Large scale shadow annotation and detection using lazy annotation and stacked CNNs. IEEE Trans Pattern Anal Mach Intell 43(4):1337–1351. https://doi.org/10.1109/TPAMI.2019.2948011

    Article  Google Scholar 

  26. Hu X, Jiang Y, Fu C, Heng P (2019) Mask-ShadowGAN: learning to remove shadows from unpaired data. In: 2019 IEEE/CVF international conference on computer vision (ICCV), pp 2472–2481. https://doi.org/10.1109/ICCV.2019.00256

  27. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T et al (2017) Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861

  28. Sandler M, Howard A, Zhu M, Zhmoginov A, Chen L (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR), Salt Lake City, UT, pp 4510–4520

  29. Howard A, Sandler M, Chu G, Liang CC, Chen B, Tan MX, Wang W, Zhu Y, Pang R, Vasudevan V, Le Q, Adam H (2019) Searching for MobileNetV3. arXiv:1905.02244

  30. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFs. Comput Sci 4:357–361

    Google Scholar 

  31. Chen L, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2016) DeepLab: semantic image segmentation with deep convolutional nets, Atrous convolution, and fully connected CRFs. In: IEEE conference on computer vision and pattern recognition (CVPR)

  32. Chen L, Papandreou G, Schroff F, Adam H (2017) Rethinking Atrous convolution for semantic image segmentation. arXiv: 1706.05587

  33. K. He, G. Gkioxari, P. Dollár and R. Girshick, Mask R-CNN, 2017 IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 2980–2988.

  34. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR)

  35. Li D, Wang S, Tang X-S, Kong W, Shi G, Chen Y (2020) Double-stream atrous network for shadow detection. Neurocomputing 417:167–175

    Article  Google Scholar 

  36. G-S Xia, Bai X, Ding J et al (2018) DOTA: a large-scale dataset for object DeTection in aerial images. https://captain-whu.github.io/DOTA/index.html

  37. Maggiori E, Tarabalka Y, Charpiat G, Alliez P (2017) Can semantic labeling methods generalize to any city? The Inria aerial image labeling benchmark. In: IEEE international geoscience and remote sensing symposium (IGARSS). 2017.

  38. Isola P, Zhu J, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, pp 5967–5976. https://doi.org/10.1109/CVPR.2017.632

  39. Pohlen T, Hermans A, Mathias M, Leibe B (2017) Full-resolution residual networks for semantic segmentation in street scenes. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), Honolulu, HI, pp 3309–3318

  40. Freitas V, Reis B, Tommaselli A (2017) Automatic shadow detection in aerial and terrestrial images. Boletim de Ciências Geodésicas 23:578–590

    Article  Google Scholar 

  41. Silva GF, Carneiro GB, Doth R, Amaral LA, de Azevedo DFG (2018) Near real-time shadow detection and removal in aerial motion imagery application. ISPRS J Photogramm Remote Sens 140:104–121

    Article  Google Scholar 

Download references

Funding

This work was supported in part by Shanghai Rising-Star Program under Grant 21QA1400100, National Natural Science Foundation of China under Grant 62176052, Shanghai Natural Science Foundation under Grant 20ZR1400800, the Shanghai Sailing Program under Grant 20YF1401600, and in part by the Fundamental Research Funds for the Central Universities of China under Grant 2232020D-47.

Author information

Author notes

  1. Dawei Li and Sifan Wang contributed equally to this paper.

Authors and Affiliations

  1. College of Information Sciences and Technology, Donghua University, Shanghai, 201620, China

    Dawei Li, Shiyu Xiang, Jinsheng Li, Yanping Yang & Xue-Song Tang

  2. Engineering Research Center of Digitized Textile and Fashion Technology, Ministry of Education, Donghua University, Shanghai, 201620, China

    Dawei Li, Shiyu Xiang, Jinsheng Li, Yanping Yang & Xue-Song Tang

  3. ByteDance Ltd., Shanghai, 200030, China

    Sifan Wang

Authors
  1. Dawei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sifan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shiyu Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jinsheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanping Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Xue-Song Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yanping Yang.

Ethics declarations

Conflict of interest

The authors declared no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, D., Wang, S., Xiang, S. et al. Dual-stream shadow detection network: biologically inspired shadow detection for remote sensing images. Neural Comput & Applic 34, 10039–10049 (2022). https://doi.org/10.1007/s00521-022-06989-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-06989-w

Keywords

Search

Navigation