Skip to main content

Advertisement

Log in

Boosting sparsely annotated shadow detection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Sparsely annotated image segmentation has gained popularity due to its ability to significantly reduce the labeling burden on training data. However, existing methods still struggle to learn complete object structures, especially for complex shadow objects. This paper discusses two prevalent issues existing in previous methods, i.e., generating noisy pseudo labels and misdetecting ambiguous regions. To tackle these challenges, we propose a novel weakly-supervised learning framework to boost sparsely annotated shadow detection. Concretely, a reliable label propagation (RLP) scheme is first designed to diffuse sparse annotations into unlabeled regions, thereby generating denser pseudo shadow masks. This scheme effectively reduces the number of noisy labels by incorporating uncertainty analysis. Then, a multi-cue semantic calibration (MSC) strategy is presented to refine the semantic features extracted from the backbone by employing edge, global, and adjacent priors. Embedded with MSC, the detection network becomes more discriminative against ambiguous regions. By combining RLP and MSC, the proposed weakly-supervised framework can detect complete and accurate shadow regions from sparse annotations. Experimental results on three benchmark datasets demonstrate that our method achieves comparable performance to recent fully-supervised methods, while requiring only about 4.5% of the pixels to be labeled.

Graphical abstract

Boosting sparsely annotated shadow detection

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Algorithm 1
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability and access

The data generated during and/or analysed during the current study are available from the first or corresponding author on reasonable request.

References

  1. Abbaszadeh Shahri A, Maghsoudi Moud F (2021) Landslide susceptibility mapping using hybridized block modular intelligence model. Bull Eng Geol Enviro 80:267–284. https://doi.org/10.1007/s10064-020-01922-8

    Article  Google Scholar 

  2. Abbaszadeh Shahri A, Shan C, Larsson S (2022) A novel approach to uncertainty quantification in groundwater table modeling by automated predictive deep learning. Nat Resour Res 31(3):1351–1373. https://doi.org/10.1007/s11053-022-10051-w

    Article  Google Scholar 

  3. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282. https://doi.org/10.1109/TPAMI.2012.120

    Article  Google Scholar 

  4. Arbel E, Hel-Or H (2010) Shadow removal using intensity surfaces and texture anchor points. IEEE Trans Pattern Anal Mach Intell 33(6):1202–1216. https://doi.org/10.1109/TPAMI.2010.157

    Article  Google Scholar 

  5. Chandrakar R, Raja R, Miri R, Sinha U, Kushwaha AKS, Raja H (2022) Enhanced the moving object detection and object tracking for traffic surveillance using rbf-fdlnn and cbf algorithm. Expert Syst App 191:116306. https://doi.org/10.1016/j.eswa.2021.116306

  6. Chen XD, Wu W, Yang W, Qin H, Wu X, Mao X (2023) Make segment anything model perfect on shadow detection. IEEE Trans Geosci Remote Sens 61:1–13. https://doi.org/10.1109/TGRS.2023.3332257

    Article  Google Scholar 

  7. Chen Y, Liu L, Phonevilay V, Gu K, Xia R, Xie J, Zhang Q, Yang K (2021) Image super-resolution reconstruction based on feature map attention mechanism. Appl Intell 51:4367–4380. https://doi.org/10.1007/s10489-020-02116-1

    Article  Google Scholar 

  8. Chen Z, Zhu L, Wan L, Wang S, Feng W, Heng PA (2020) A multi-task mean teacher for semi-supervised shadow detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5611–5620. https://doi.org/10.1109/CVPR42600.2020.00565

  9. Cui J, Liu S, Tian Z, Zhong Z, Jia J (2022) Reslt: Residual learning for long-tailed recognition. IEEE Trans Pattern Anal Mach Intell 45(3):3695–3706. https://doi.org/10.1109/TPAMI.2022.3174892

    Article  Google Scholar 

  10. Dhamija T, Gupta A, Gupta S, Anjum Katarya R, Singh G (2023) Semantic segmentation in medical images through transfused convolution and transformer networks. Appl Intell 53(1):1132–1148. https://doi.org/10.1007/s10489-022-03642-w

    Article  Google Scholar 

  11. Gao G, Xu G, Yu Y, Xie J, Yang J, Yue D (2021) MSCFNet: A lightweight network with multi-scale context fusion for real-time semantic segmentation. IEEE Trans Intell Transport Syst 23(12):25489–25499. https://doi.org/10.1109/TITS.2021.3098355

    Article  Google Scholar 

  12. Gao S, Zhang W, Wang Y, Guo Q, Zhang C, He Y, Zhang W (2022) Weakly-supervised salient object detection using point supervison. Proceedings of the AAAI conference on artificial intelligence 36:670–678. https://doi.org/10.1609/aaai.v32i1.12308

    Article  Google Scholar 

  13. Ghaderi A, Abbaszadeh Shahri A, Larsson S (2022) A visualized hybrid intelligent model to delineate swedish fine-grained soil layers using clay sensitivity. Catena 214:106289. https://doi.org/10.1016/j.catena.2022.106289

    Article  Google Scholar 

  14. Guo R, Dai Q, Hoiem D (2012) Paired regions for shadow detection and removal. IEEE Trans Pattern Anal Mach Intell 35(12):2956–2967. https://doi.org/10.1109/TPAMI.2012.214

    Article  Google Scholar 

  15. He R, Dong Q, Lin J, Lau RW (2023) Weakly-supervised camouflaged object detection with scribble annotations. Proceedings of the AAAI conference on artificial intelligence pp 1–12. https://doi.org/10.1609/aaai.v37i1.25156

  16. Hou L, Vicente TFY, Hoai M, Samaras D (2019) Large scale shadow annotation and detection using lazy annotation and stacked cnns. IEEE Trans Pattern Anal Mach Intell 43(4):1337–1351. https://doi.org/10.1109/TPAMI.2019.2948011

    Article  Google Scholar 

  17. Hu X, Fu CW, Zhu L, Qin J, Heng PA (2019) Direction-aware spatial context features for shadow detection and removal. IEEE Trans Pattern Anal Mach Intell 42(11):2795–2808. https://doi.org/10.1109/TPAMI.2019.2919616

    Article  Google Scholar 

  18. Hu X, Wang T, Fu CW, Jiang Y, Wang Q, Heng PA (2021) Revisiting shadow detection: A new benchmark dataset for complex world. IEEE Trans Image Process 30:1925–1934. https://doi.org/10.1109/TIP.2021.3049331

    Article  Google Scholar 

  19. Jie L, Zhang H (2023) RMLANet: Random multi-level attention network for shadow detection and removal. IEEE Trans Circ Syst Vid Technol 33(12):7819–7831. https://doi.org/10.1109/TCSVT.2023.3283416

    Article  Google Scholar 

  20. Khan SH, Bennamoun M, Sohel F, Togneri R (2015) Automatic shadow detection and removal from a single image. IEEE Trans Pattern Anal Mach Intell 38(3):431–446. https://doi.org/10.1109/TPAMI.2015.2462355

    Article  Google Scholar 

  21. Kho S, Lee P, Lee W, Ki M, Byun H (2022) Exploiting shape cues for weakly supervised semantic segmentation. Pattern Recogn 132:108953. https://doi.org/10.1016/j.patcog.2022.108953

  22. Le H, Vicente TFY, Nguyen V, Hoai M, Samaras D (2018) A+D Net: Training a shadow detector with adversarial shadow attenuation. In: Proceedings of the European conference on computer vision, pp 662–678. https://doi.org/10.1007/978-3-030-01216-8-41

  23. Liang J, Hu D, Wang Y, He R, Feng J (2021) Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Trans Pattern Anal Mach Intell 44(11):8602–8617. https://doi.org/10.1109/TPAMI.2021.3103390

    Article  Google Scholar 

  24. Liang X, Liu Z, Wang J, Jin X, Du Z (2023) Uncertainty quantification-based robust deep learning for building energy systems considering distribution shift problem. Appl Energy 337:120889. https://doi.org/10.1016/j.apenergy.2023.120889

    Article  Google Scholar 

  25. Liu Y, Wang P, Cao Y, Liang Z, Lau RW (2021) Weakly-supervised salient object detection with saliency bounding boxes. IEEE Trans Image Process 30:4423–4435. https://doi.org/10.1109/TIP.2021.3071691

    Article  Google Scholar 

  26. Ma L, Luo X, Hong H, Zhang Y, Wang L, Wu J (2023) Scribble-attention hierarchical network for weakly supervised salient object detection in optical remote sensing images. Appl Intell 53(10):12999–13017. https://doi.org/10.1007/s10489-022-04014-0

    Article  Google Scholar 

  27. Mao Z, Zhou Y, Sun J, Wu H, Pan F, Ahmad B (2023) Weakly-supervised object localization with gradient-pyramid feature. Appl Intell 53(3):2923–2935. https://doi.org/10.1007/s10489-022-03686-y

    Article  Google Scholar 

  28. Mehrtash A, Wells WM, Tempany CM, Abolmaesumi P, Kapur T (2020) Confidence calibration and predictive uncertainty estimation for deep medical image segmentation. IEEE Trans Med Imaging 39(12):3868–3878. https://doi.org/10.1109/TMI.2020.3006437

    Article  Google Scholar 

  29. Nguyen V, Yago Vicente TF, Zhao M, Hoai M, Samaras D (2017) Shadow detection with conditional generative adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 4510–4518. https://doi.org/10.1109/ICCV.2017.483

  30. Pachade S, Porwal P, Kokare M, Giancardo L, Mériaudeau F (2021) NENet: Nested efficientnet and adversarial learning for joint optic disc and cup segmentation. Med Image Anal 74:102253. https://doi.org/10.1016/j.media.2021.102253

  31. Pan Z, Sun H, Jiang P, Li G, Tu C, Ling H (2024) CC4S: Encouraging certainty and consistency in scribble-supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 1–18. https://doi.org/10.1109/TPAMI.2024.3415387

  32. Piao Y, Wang J, Zhang M, Lu H (2021) Mfnet: Multi-filter directive network for weakly supervised salient object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4136–4145. https://doi.org/10.1109/ICCV48922.2021.00410

  33. Piao Y, Wu W, Zhang M, Jiang Y, Lu H (2022) Noise-sensitive adversarial learning for weakly supervised salient object detection. IEEE Trans Multimed 25:2888–2897. https://doi.org/10.1109/TMM.2022.3152567

    Article  Google Scholar 

  34. Pu M, Huang Y, Liu Y, Guan Q, Ling H (2022) Edter: Edge detection with transformer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1402–1412. https://doi.org/10.1109/CVPR52688.2022.00146

  35. Qin Z, Chen Y, Zhu G, Zhou E, Zhou Y, Zhou Y, Zhu C (2024) Enhanced pseudo-label generation with self-supervised training for weakly-supervised semantic segmentation. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2024.3364764

    Article  Google Scholar 

  36. Ren Q, Lu S, Zhang J, Hu R (2020) Salient object detection by fusing local and global contexts. IEEE Trans Multimed 23:1442–1453. https://doi.org/10.1109/TMM.2020.2997178

    Article  Google Scholar 

  37. Sahoo S, Nanda PK (2021) Adaptive feature fusion and spatio-temporal background modeling in kde framework for object detection and shadow removal. IEEE Trans Circ Syst Vid Technol 32(3):1103–1118. https://doi.org/10.1109/TCSVT.2021.3074143

    Article  Google Scholar 

  38. Sanin A, Sanderson C, Lovell BC (2012) Shadow detection: A survey and comparative evaluation of recent methods. Pattern Recogn 45(4):1684–1695. https://doi.org/10.1016/j.patcog.2011.10.001

    Article  Google Scholar 

  39. Shamsi A, Asgharnezhad H, Bouchani Z, Jahanian K, Saberi M, Wang X, Razzak I, Alizadehsani R, Mohammadi A, Alinejad-Rokny H (2023) A novel uncertainty-aware deep learning technique with an application on skin cancer diagnosis. Neural Computing and Applications 35(30):22179–22188. https://doi.org/10.1007/s00521-023-08930-1

    Article  Google Scholar 

  40. Štampfl V, Ahtik J (2024) Shadow segmentation with image thresholding for describing the harshness of light sources. IEEE Trans Image Process 33:3428–3440. https://doi.org/10.1109/TIP.2024.3403487

    Article  Google Scholar 

  41. Sun J, Xu K, Pang Y, Zhang L, Lu H, Hancke G, Lau R (2023) Adaptive illumination mapping for shadow detection in raw images. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 12709–12718. https://doi.org/10.1109/ICCV51070.2023.01167

  42. Tang Y, Wu X (2019) Salient object detection using cascaded convolutional neural networks and adversarial learning. IEEE Trans Multimed 21(9):2237–2247. https://doi.org/10.1109/TMM.2019.2900908

    Article  Google Scholar 

  43. Tian J, Qi X, Qu L, Tang Y (2016) New spectrum ratio properties and features for shadow detection. Pattern Recogn 51:85–96. https://doi.org/10.1016/j.patcog.2015.09.006

    Article  Google Scholar 

  44. Veksler O (2023) Test time adaptation with regularized loss for weakly supervised salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7360–7369. https://doi.org/10.1109/CVPR52729.2023.00711

  45. Vicente TFY, Hoai M, Samaras D (2017) Leave-one-out kernel optimization for shadow detection and removal. IEEE Trans Pattern Anal Mach Intell 40(3):682–695. https://doi.org/10.1109/TPAMI.2017.2691703

    Article  Google Scholar 

  46. Wang J, Li X, Yang J (2018) Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1788–1797. https://doi.org/10.1109/CVPR.2018.00192

  47. Wang L, Zhang X, Su H, Zhu J (2024) A comprehensive survey of continual learning: theory, method and application. IEEE Trans Pattern Anal Mach Intell 46(8):5362–5383. https://doi.org/10.1109/TPAMI.2024.3367329

    Article  Google Scholar 

  48. Wang T, Hu X, Heng PA, Fu CW (2022) Instance shadow detection with a single-stage detector. IEEE Trans Pattern Anal Mach Intell 45(3):3259–3273. https://doi.org/10.1109/TPAMI.2022.3185628

    Article  Google Scholar 

  49. Wu L, Fang L, He X, He M, Ma J, Zhong Z (2023) Querying labeled for unlabeled: Cross-image semantic consistency guided semi-supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 45(7):8827–8844. https://doi.org/10.1109/TPAMI.2022.3233584

    Article  Google Scholar 

  50. Wu XT, Wang Y, Wan Y, Wu W (2022) Annotation is easy: Learning to generate a shadow mask. Comput Graph 104:152–161. https://doi.org/10.1016/j.cag.2022.04.003

    Article  Google Scholar 

  51. Xu H, Yang M, Deng L, Qian Y, Wang C (2021) Neutral cross-entropy loss based unsupervised domain adaptation for semantic segmentation. IEEE Trans Image Process 30:4516–4525. https://doi.org/10.1109/TIP.2021.3073285

    Article  MathSciNet  Google Scholar 

  52. Xu M, Zhu J, Lv P, Zhou B, Tappen MF, Ji R (2017) Learning-based shadow recognition and removal from monochromatic natural images. IEEE Trans Image Process 26(12):5811–5824. https://doi.org/10.1109/TIP.2017.2737321

    Article  MathSciNet  Google Scholar 

  53. Xu W, Chen H, Su Q, Ji C, Xu W, Memon MS, Zhou J (2019) Shadow detection and removal in apple image segmentation under natural light conditions using an ultrametric contour map. Biosyst Eng 184:142–154. https://doi.org/10.1016/j.biosystemseng.2019.06.016

    Article  Google Scholar 

  54. Yelleni SH, Kumari D, Srijith P et al (2024) Monte carlo dropblock for modeling uncertainty in object detection. Pattern Recogn 146:110003. https://doi.org/10.1016/j.patcog.2023.110003

  55. Younas F, Usman M, Yan WQ (2023) A deep ensemble learning method for colorectal polyp classification with optimized network parameters. Appl Intell 53(2):2410–2433. https://doi.org/10.1007/s10489-022-03689-9

    Article  Google Scholar 

  56. Yu S, Zhang B, Xiao J, Lim EG (2021) Structure-consistent weakly supervised salient object detection with local saliency coherence. Proceedings of the AAAI conference on artificial intelligence 35:3234–3242. https://doi.org/10.1609/aaai.v35i4.16434

    Article  Google Scholar 

  57. Yu Y, Guan H, Li D, Gu T, Tang E, Li A (2020) Orientation guided anchoring for geospatial object detection from remote sensing imagery. ISPRS J Photogramm Remote Sens 160:67–82. https://doi.org/10.1016/j.isprsjprs.2019.12.001

    Article  Google Scholar 

  58. Zhang B, Xiao J, Jiao J, Wei Y, Zhao Y (2021) Affinity attention graph neural network for weakly supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 44(11):8082–8096. https://doi.org/10.1109/TPAMI.2021.3083269

    Article  Google Scholar 

  59. Zhang J, Yu X, Li A, Song P, Liu B, Dai Y (2020) Weakly-supervised salient object detection via scribble annotations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12546–12555. https://doi.org/10.1109/CVPR42600.2020.01256

  60. Zhang K, Zhuang X (2022) Cyclemix: A holistic strategy for medical image segmentation from scribble supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11656–11665. https://doi.org/10.1109/CVPR52688.2022.01136

  61. Zhang P, Liu W, Zeng Y, Lei Y, Lu H (2021) Looking for the detail and context devils: High-resolution salient object detection. IEEE Trans Image Process 30:3204–3216. https://doi.org/10.1109/TIP.2020.3045624

    Article  Google Scholar 

  62. Zhang X, Barron JT, Tsai YT, Pandey R, Zhang X, Ng R, Jacobs DE (2020) Portrait shadow manipulation. ACM Trans Graphics 39(4):78–1. https://doi.org/10.1145/3386569.3392390

    Article  Google Scholar 

  63. Zhao T, Yin Z (2020) Weakly supervised cell segmentation by point annotation. IEEE Trans Med Imaging 40(10):2736–2747. https://doi.org/10.1109/TMI.2020.3046292

    Article  Google Scholar 

  64. Zheng Q, Qiao X, Cao Y, Lau RW (2019) Distraction-aware shadow detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5167–5176. https://doi.org/10.1109/CVPR.2019.00531

  65. Zhou C, Cui Z, Xu C, Han C, Yang J (2023) Exploratory inference learning for scribble supervised semantic segmentation. Proceedings of the AAAI conference on artificial intelligence 37:3760–3768. https://doi.org/10.1609/aaai.v37i3.25488

    Article  Google Scholar 

  66. Zhu L, Deng Z, Hu X, Fu CW, Xu X, Qin J, Heng PA (2018) Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In: Proceedings of the European conference on computer vision, pp 121–136. https://doi.org/10.1007/978-3-030-01231-1-8

  67. Zhu L, Xu K, Ke Z, Lau, RW (2021) Mitigating intensity bias in shadow detection via feature decomposition and reweighting. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4702–4711. https://doi.org/10.1109/CVPR42600.2020.00565

  68. Zhu Y, Fu X, Cao C, Wang X, Sun Q, Zha ZJ (2022) Single image shadow detection via complementary mechanism. In: Proceedings of the ACM international conference on multimedia, pp 6717–6726. https://doi.org/10.1145/3503161.3547904

  69. Zou BJ, Guo YD, He Q, Ouyang PB, Liu K, Chen ZL (2018) 3d filtering by block matching and convolutional neural network for image denoising. J Comput Sci Technol 33:838–848. https://doi.org/10.1007/s11390-018-1859-7

Download references

Funding

This work was supported by the National Natural Science Foundation of China [grant number 61972121] and the Zhejiang Provincial Natural Science Foundation [grant number LQZSZ24E050001].

Author information

Authors and Affiliations

Authors

Contributions

Kai Zhou: Conceptualization, Methodology, Writing - Original Draft, Writing - Review & Editing. Yanli Shao: Methodology, Visualization, Validation. Jinglong Fang: Supervision, Funding acquisition. Dan Wei: Project administration. Wanlu Sun: Formal analysis, Data Curation.

Corresponding author

Correspondence to Yanli Shao.

Ethics declarations

Conflicts of interest

The authors declare that they have no conflict of interest.

Ethics approval

The work follows appropriate ethical standards. The proposed model trained with publicly available data, for which no ethical approval was required.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhou, K., Shao, Y., Fang, J. et al. Boosting sparsely annotated shadow detection. Appl Intell 54, 10541–10560 (2024). https://doi.org/10.1007/s10489-024-05740-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05740-3

Keywords

Navigation