Abstract
Sparsely annotated image segmentation has gained popularity due to its ability to significantly reduce the labeling burden on training data. However, existing methods still struggle to learn complete object structures, especially for complex shadow objects. This paper discusses two prevalent issues existing in previous methods, i.e., generating noisy pseudo labels and misdetecting ambiguous regions. To tackle these challenges, we propose a novel weakly-supervised learning framework to boost sparsely annotated shadow detection. Concretely, a reliable label propagation (RLP) scheme is first designed to diffuse sparse annotations into unlabeled regions, thereby generating denser pseudo shadow masks. This scheme effectively reduces the number of noisy labels by incorporating uncertainty analysis. Then, a multi-cue semantic calibration (MSC) strategy is presented to refine the semantic features extracted from the backbone by employing edge, global, and adjacent priors. Embedded with MSC, the detection network becomes more discriminative against ambiguous regions. By combining RLP and MSC, the proposed weakly-supervised framework can detect complete and accurate shadow regions from sparse annotations. Experimental results on three benchmark datasets demonstrate that our method achieves comparable performance to recent fully-supervised methods, while requiring only about 4.5% of the pixels to be labeled.
Graphical abstract
Boosting sparsely annotated shadow detection
![](http://media.springernature.com/lw685/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Figf_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig6_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig7_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Figg_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig8_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig9_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig10_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig11_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig12_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig13_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig14_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig15_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig16_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10489-024-05740-3/MediaObjects/10489_2024_5740_Fig17_HTML.png)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability and access
The data generated during and/or analysed during the current study are available from the first or corresponding author on reasonable request.
References
Abbaszadeh Shahri A, Maghsoudi Moud F (2021) Landslide susceptibility mapping using hybridized block modular intelligence model. Bull Eng Geol Enviro 80:267–284. https://doi.org/10.1007/s10064-020-01922-8
Abbaszadeh Shahri A, Shan C, Larsson S (2022) A novel approach to uncertainty quantification in groundwater table modeling by automated predictive deep learning. Nat Resour Res 31(3):1351–1373. https://doi.org/10.1007/s11053-022-10051-w
Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282. https://doi.org/10.1109/TPAMI.2012.120
Arbel E, Hel-Or H (2010) Shadow removal using intensity surfaces and texture anchor points. IEEE Trans Pattern Anal Mach Intell 33(6):1202–1216. https://doi.org/10.1109/TPAMI.2010.157
Chandrakar R, Raja R, Miri R, Sinha U, Kushwaha AKS, Raja H (2022) Enhanced the moving object detection and object tracking for traffic surveillance using rbf-fdlnn and cbf algorithm. Expert Syst App 191:116306. https://doi.org/10.1016/j.eswa.2021.116306
Chen XD, Wu W, Yang W, Qin H, Wu X, Mao X (2023) Make segment anything model perfect on shadow detection. IEEE Trans Geosci Remote Sens 61:1–13. https://doi.org/10.1109/TGRS.2023.3332257
Chen Y, Liu L, Phonevilay V, Gu K, Xia R, Xie J, Zhang Q, Yang K (2021) Image super-resolution reconstruction based on feature map attention mechanism. Appl Intell 51:4367–4380. https://doi.org/10.1007/s10489-020-02116-1
Chen Z, Zhu L, Wan L, Wang S, Feng W, Heng PA (2020) A multi-task mean teacher for semi-supervised shadow detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5611–5620. https://doi.org/10.1109/CVPR42600.2020.00565
Cui J, Liu S, Tian Z, Zhong Z, Jia J (2022) Reslt: Residual learning for long-tailed recognition. IEEE Trans Pattern Anal Mach Intell 45(3):3695–3706. https://doi.org/10.1109/TPAMI.2022.3174892
Dhamija T, Gupta A, Gupta S, Anjum Katarya R, Singh G (2023) Semantic segmentation in medical images through transfused convolution and transformer networks. Appl Intell 53(1):1132–1148. https://doi.org/10.1007/s10489-022-03642-w
Gao G, Xu G, Yu Y, Xie J, Yang J, Yue D (2021) MSCFNet: A lightweight network with multi-scale context fusion for real-time semantic segmentation. IEEE Trans Intell Transport Syst 23(12):25489–25499. https://doi.org/10.1109/TITS.2021.3098355
Gao S, Zhang W, Wang Y, Guo Q, Zhang C, He Y, Zhang W (2022) Weakly-supervised salient object detection using point supervison. Proceedings of the AAAI conference on artificial intelligence 36:670–678. https://doi.org/10.1609/aaai.v32i1.12308
Ghaderi A, Abbaszadeh Shahri A, Larsson S (2022) A visualized hybrid intelligent model to delineate swedish fine-grained soil layers using clay sensitivity. Catena 214:106289. https://doi.org/10.1016/j.catena.2022.106289
Guo R, Dai Q, Hoiem D (2012) Paired regions for shadow detection and removal. IEEE Trans Pattern Anal Mach Intell 35(12):2956–2967. https://doi.org/10.1109/TPAMI.2012.214
He R, Dong Q, Lin J, Lau RW (2023) Weakly-supervised camouflaged object detection with scribble annotations. Proceedings of the AAAI conference on artificial intelligence pp 1–12. https://doi.org/10.1609/aaai.v37i1.25156
Hou L, Vicente TFY, Hoai M, Samaras D (2019) Large scale shadow annotation and detection using lazy annotation and stacked cnns. IEEE Trans Pattern Anal Mach Intell 43(4):1337–1351. https://doi.org/10.1109/TPAMI.2019.2948011
Hu X, Fu CW, Zhu L, Qin J, Heng PA (2019) Direction-aware spatial context features for shadow detection and removal. IEEE Trans Pattern Anal Mach Intell 42(11):2795–2808. https://doi.org/10.1109/TPAMI.2019.2919616
Hu X, Wang T, Fu CW, Jiang Y, Wang Q, Heng PA (2021) Revisiting shadow detection: A new benchmark dataset for complex world. IEEE Trans Image Process 30:1925–1934. https://doi.org/10.1109/TIP.2021.3049331
Jie L, Zhang H (2023) RMLANet: Random multi-level attention network for shadow detection and removal. IEEE Trans Circ Syst Vid Technol 33(12):7819–7831. https://doi.org/10.1109/TCSVT.2023.3283416
Khan SH, Bennamoun M, Sohel F, Togneri R (2015) Automatic shadow detection and removal from a single image. IEEE Trans Pattern Anal Mach Intell 38(3):431–446. https://doi.org/10.1109/TPAMI.2015.2462355
Kho S, Lee P, Lee W, Ki M, Byun H (2022) Exploiting shape cues for weakly supervised semantic segmentation. Pattern Recogn 132:108953. https://doi.org/10.1016/j.patcog.2022.108953
Le H, Vicente TFY, Nguyen V, Hoai M, Samaras D (2018) A+D Net: Training a shadow detector with adversarial shadow attenuation. In: Proceedings of the European conference on computer vision, pp 662–678. https://doi.org/10.1007/978-3-030-01216-8-41
Liang J, Hu D, Wang Y, He R, Feng J (2021) Source data-absent unsupervised domain adaptation through hypothesis transfer and labeling transfer. IEEE Trans Pattern Anal Mach Intell 44(11):8602–8617. https://doi.org/10.1109/TPAMI.2021.3103390
Liang X, Liu Z, Wang J, Jin X, Du Z (2023) Uncertainty quantification-based robust deep learning for building energy systems considering distribution shift problem. Appl Energy 337:120889. https://doi.org/10.1016/j.apenergy.2023.120889
Liu Y, Wang P, Cao Y, Liang Z, Lau RW (2021) Weakly-supervised salient object detection with saliency bounding boxes. IEEE Trans Image Process 30:4423–4435. https://doi.org/10.1109/TIP.2021.3071691
Ma L, Luo X, Hong H, Zhang Y, Wang L, Wu J (2023) Scribble-attention hierarchical network for weakly supervised salient object detection in optical remote sensing images. Appl Intell 53(10):12999–13017. https://doi.org/10.1007/s10489-022-04014-0
Mao Z, Zhou Y, Sun J, Wu H, Pan F, Ahmad B (2023) Weakly-supervised object localization with gradient-pyramid feature. Appl Intell 53(3):2923–2935. https://doi.org/10.1007/s10489-022-03686-y
Mehrtash A, Wells WM, Tempany CM, Abolmaesumi P, Kapur T (2020) Confidence calibration and predictive uncertainty estimation for deep medical image segmentation. IEEE Trans Med Imaging 39(12):3868–3878. https://doi.org/10.1109/TMI.2020.3006437
Nguyen V, Yago Vicente TF, Zhao M, Hoai M, Samaras D (2017) Shadow detection with conditional generative adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 4510–4518. https://doi.org/10.1109/ICCV.2017.483
Pachade S, Porwal P, Kokare M, Giancardo L, Mériaudeau F (2021) NENet: Nested efficientnet and adversarial learning for joint optic disc and cup segmentation. Med Image Anal 74:102253. https://doi.org/10.1016/j.media.2021.102253
Pan Z, Sun H, Jiang P, Li G, Tu C, Ling H (2024) CC4S: Encouraging certainty and consistency in scribble-supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 1–18. https://doi.org/10.1109/TPAMI.2024.3415387
Piao Y, Wang J, Zhang M, Lu H (2021) Mfnet: Multi-filter directive network for weakly supervised salient object detection. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4136–4145. https://doi.org/10.1109/ICCV48922.2021.00410
Piao Y, Wu W, Zhang M, Jiang Y, Lu H (2022) Noise-sensitive adversarial learning for weakly supervised salient object detection. IEEE Trans Multimed 25:2888–2897. https://doi.org/10.1109/TMM.2022.3152567
Pu M, Huang Y, Liu Y, Guan Q, Ling H (2022) Edter: Edge detection with transformer. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1402–1412. https://doi.org/10.1109/CVPR52688.2022.00146
Qin Z, Chen Y, Zhu G, Zhou E, Zhou Y, Zhou Y, Zhu C (2024) Enhanced pseudo-label generation with self-supervised training for weakly-supervised semantic segmentation. IEEE Transactions on Circuits and Systems for Video Technology. https://doi.org/10.1109/TCSVT.2024.3364764
Ren Q, Lu S, Zhang J, Hu R (2020) Salient object detection by fusing local and global contexts. IEEE Trans Multimed 23:1442–1453. https://doi.org/10.1109/TMM.2020.2997178
Sahoo S, Nanda PK (2021) Adaptive feature fusion and spatio-temporal background modeling in kde framework for object detection and shadow removal. IEEE Trans Circ Syst Vid Technol 32(3):1103–1118. https://doi.org/10.1109/TCSVT.2021.3074143
Sanin A, Sanderson C, Lovell BC (2012) Shadow detection: A survey and comparative evaluation of recent methods. Pattern Recogn 45(4):1684–1695. https://doi.org/10.1016/j.patcog.2011.10.001
Shamsi A, Asgharnezhad H, Bouchani Z, Jahanian K, Saberi M, Wang X, Razzak I, Alizadehsani R, Mohammadi A, Alinejad-Rokny H (2023) A novel uncertainty-aware deep learning technique with an application on skin cancer diagnosis. Neural Computing and Applications 35(30):22179–22188. https://doi.org/10.1007/s00521-023-08930-1
Štampfl V, Ahtik J (2024) Shadow segmentation with image thresholding for describing the harshness of light sources. IEEE Trans Image Process 33:3428–3440. https://doi.org/10.1109/TIP.2024.3403487
Sun J, Xu K, Pang Y, Zhang L, Lu H, Hancke G, Lau R (2023) Adaptive illumination mapping for shadow detection in raw images. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 12709–12718. https://doi.org/10.1109/ICCV51070.2023.01167
Tang Y, Wu X (2019) Salient object detection using cascaded convolutional neural networks and adversarial learning. IEEE Trans Multimed 21(9):2237–2247. https://doi.org/10.1109/TMM.2019.2900908
Tian J, Qi X, Qu L, Tang Y (2016) New spectrum ratio properties and features for shadow detection. Pattern Recogn 51:85–96. https://doi.org/10.1016/j.patcog.2015.09.006
Veksler O (2023) Test time adaptation with regularized loss for weakly supervised salient object detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7360–7369. https://doi.org/10.1109/CVPR52729.2023.00711
Vicente TFY, Hoai M, Samaras D (2017) Leave-one-out kernel optimization for shadow detection and removal. IEEE Trans Pattern Anal Mach Intell 40(3):682–695. https://doi.org/10.1109/TPAMI.2017.2691703
Wang J, Li X, Yang J (2018) Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1788–1797. https://doi.org/10.1109/CVPR.2018.00192
Wang L, Zhang X, Su H, Zhu J (2024) A comprehensive survey of continual learning: theory, method and application. IEEE Trans Pattern Anal Mach Intell 46(8):5362–5383. https://doi.org/10.1109/TPAMI.2024.3367329
Wang T, Hu X, Heng PA, Fu CW (2022) Instance shadow detection with a single-stage detector. IEEE Trans Pattern Anal Mach Intell 45(3):3259–3273. https://doi.org/10.1109/TPAMI.2022.3185628
Wu L, Fang L, He X, He M, Ma J, Zhong Z (2023) Querying labeled for unlabeled: Cross-image semantic consistency guided semi-supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 45(7):8827–8844. https://doi.org/10.1109/TPAMI.2022.3233584
Wu XT, Wang Y, Wan Y, Wu W (2022) Annotation is easy: Learning to generate a shadow mask. Comput Graph 104:152–161. https://doi.org/10.1016/j.cag.2022.04.003
Xu H, Yang M, Deng L, Qian Y, Wang C (2021) Neutral cross-entropy loss based unsupervised domain adaptation for semantic segmentation. IEEE Trans Image Process 30:4516–4525. https://doi.org/10.1109/TIP.2021.3073285
Xu M, Zhu J, Lv P, Zhou B, Tappen MF, Ji R (2017) Learning-based shadow recognition and removal from monochromatic natural images. IEEE Trans Image Process 26(12):5811–5824. https://doi.org/10.1109/TIP.2017.2737321
Xu W, Chen H, Su Q, Ji C, Xu W, Memon MS, Zhou J (2019) Shadow detection and removal in apple image segmentation under natural light conditions using an ultrametric contour map. Biosyst Eng 184:142–154. https://doi.org/10.1016/j.biosystemseng.2019.06.016
Yelleni SH, Kumari D, Srijith P et al (2024) Monte carlo dropblock for modeling uncertainty in object detection. Pattern Recogn 146:110003. https://doi.org/10.1016/j.patcog.2023.110003
Younas F, Usman M, Yan WQ (2023) A deep ensemble learning method for colorectal polyp classification with optimized network parameters. Appl Intell 53(2):2410–2433. https://doi.org/10.1007/s10489-022-03689-9
Yu S, Zhang B, Xiao J, Lim EG (2021) Structure-consistent weakly supervised salient object detection with local saliency coherence. Proceedings of the AAAI conference on artificial intelligence 35:3234–3242. https://doi.org/10.1609/aaai.v35i4.16434
Yu Y, Guan H, Li D, Gu T, Tang E, Li A (2020) Orientation guided anchoring for geospatial object detection from remote sensing imagery. ISPRS J Photogramm Remote Sens 160:67–82. https://doi.org/10.1016/j.isprsjprs.2019.12.001
Zhang B, Xiao J, Jiao J, Wei Y, Zhao Y (2021) Affinity attention graph neural network for weakly supervised semantic segmentation. IEEE Trans Pattern Anal Mach Intell 44(11):8082–8096. https://doi.org/10.1109/TPAMI.2021.3083269
Zhang J, Yu X, Li A, Song P, Liu B, Dai Y (2020) Weakly-supervised salient object detection via scribble annotations. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 12546–12555. https://doi.org/10.1109/CVPR42600.2020.01256
Zhang K, Zhuang X (2022) Cyclemix: A holistic strategy for medical image segmentation from scribble supervision. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11656–11665. https://doi.org/10.1109/CVPR52688.2022.01136
Zhang P, Liu W, Zeng Y, Lei Y, Lu H (2021) Looking for the detail and context devils: High-resolution salient object detection. IEEE Trans Image Process 30:3204–3216. https://doi.org/10.1109/TIP.2020.3045624
Zhang X, Barron JT, Tsai YT, Pandey R, Zhang X, Ng R, Jacobs DE (2020) Portrait shadow manipulation. ACM Trans Graphics 39(4):78–1. https://doi.org/10.1145/3386569.3392390
Zhao T, Yin Z (2020) Weakly supervised cell segmentation by point annotation. IEEE Trans Med Imaging 40(10):2736–2747. https://doi.org/10.1109/TMI.2020.3046292
Zheng Q, Qiao X, Cao Y, Lau RW (2019) Distraction-aware shadow detection. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5167–5176. https://doi.org/10.1109/CVPR.2019.00531
Zhou C, Cui Z, Xu C, Han C, Yang J (2023) Exploratory inference learning for scribble supervised semantic segmentation. Proceedings of the AAAI conference on artificial intelligence 37:3760–3768. https://doi.org/10.1609/aaai.v37i3.25488
Zhu L, Deng Z, Hu X, Fu CW, Xu X, Qin J, Heng PA (2018) Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In: Proceedings of the European conference on computer vision, pp 121–136. https://doi.org/10.1007/978-3-030-01231-1-8
Zhu L, Xu K, Ke Z, Lau, RW (2021) Mitigating intensity bias in shadow detection via feature decomposition and reweighting. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 4702–4711. https://doi.org/10.1109/CVPR42600.2020.00565
Zhu Y, Fu X, Cao C, Wang X, Sun Q, Zha ZJ (2022) Single image shadow detection via complementary mechanism. In: Proceedings of the ACM international conference on multimedia, pp 6717–6726. https://doi.org/10.1145/3503161.3547904
Zou BJ, Guo YD, He Q, Ouyang PB, Liu K, Chen ZL (2018) 3d filtering by block matching and convolutional neural network for image denoising. J Comput Sci Technol 33:838–848. https://doi.org/10.1007/s11390-018-1859-7
Funding
This work was supported by the National Natural Science Foundation of China [grant number 61972121] and the Zhejiang Provincial Natural Science Foundation [grant number LQZSZ24E050001].
Author information
Authors and Affiliations
Contributions
Kai Zhou: Conceptualization, Methodology, Writing - Original Draft, Writing - Review & Editing. Yanli Shao: Methodology, Visualization, Validation. Jinglong Fang: Supervision, Funding acquisition. Dan Wei: Project administration. Wanlu Sun: Formal analysis, Data Curation.
Corresponding author
Ethics declarations
Conflicts of interest
The authors declare that they have no conflict of interest.
Ethics approval
The work follows appropriate ethical standards. The proposed model trained with publicly available data, for which no ethical approval was required.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Zhou, K., Shao, Y., Fang, J. et al. Boosting sparsely annotated shadow detection. Appl Intell 54, 10541–10560 (2024). https://doi.org/10.1007/s10489-024-05740-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-024-05740-3