Skip to main content
SpringerLink
Log in

Don’t worry about noisy labels in soft shadow detection

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Soft shadow is harder to detect than hard shadow as its complex characteristics (i.e., low-contrast, irregular shape, and ambiguous shadow boundaries). To improve the detecting capacity of these images, in this paper, we create a new benchmark for soft shadow detection and then design a reasonable supervision strategy to alleviate the effect of annotation noises. Next, we present a general shadow detection framework based on transformer to deal with complex scenes. Concretely, we combine the traditional channel attention and recent popular self-attention into our network. Moreover, we introduce a deep supervision mechanism that performs deep layer supervision to “guide” early classification results at each layer, which can further improve our detection performance. Finally, experimental results on three datasets show that our shadow transformer can be favorable against current state-of-the-art detectors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The data that support the findings of this study are available from Xian-Tao Wu (xiantao.cs@gmail.com) on reasonable request.

References

  1. Banerjee, A., Das, N., Santosh, K.: Weber local descriptor for image analysis and recognition: a survey. Vis. Comput. 38, 1–23 (2020)

    Google Scholar 

  2. Brendel, W., Bethge, M.: Approximating CNNS with bag-of-local-features models works surprisingly well on imagenet (2019). arXiv preprint arXiv:1904.00760

  3. Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., Zagoruyko, S.: End-to-end object detection with transformers. In: European Conference on Computer Vision, pp. 213–229. Springer (2020)

  4. Chen, J., Lu, Y., Yu, Q., Luo, X., Adeli, E., Wang, Y., Lu, L., Yuille, A.L., Zhou, Y.: Transunet: transformers make strong encoders for medical image segmentation (2021). arXiv preprint arXiv:2102.04306

  5. Chen, Z., Zhu, L., Wan, L., Wang, S., Feng, W., Heng, P.A.: A multi-task mean teacher for semi-supervised shadow detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5611–5620 (2020)

  6. Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., et al.: An image is worth 16x16 words: transformers for image recognition at scale (2020). arXiv preprint arXiv:2010.11929

  7. Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. Int. J. Comput. Vis. 85(1), 35–57 (2009)

    Article  Google Scholar 

  8. Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 59–68 (2005)

    Article  Google Scholar 

  9. Gryka, M., Terry, M., Brostow, G.J.: Learning to remove soft shadows. ACM Trans. Graph. TOG 34(5), 1–15 (2015)

    Article  Google Scholar 

  10. Guo, R., Dai, Q., Hoiem, D.: Single-image shadow detection and removal using paired regions. In: CVPR 2011, pp. 2033–2040. IEEE (2011)

  11. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  12. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7132–7141 (2018)

  13. Hu, X., Wang, T., Fu, C.W., Jiang, Y., Wang, Q., Heng, P.A.: Revisiting shadow detection: a new benchmark dataset for complex world. IEEE Trans. Image Process. 30, 1925–1934 (2021)

    Article  Google Scholar 

  14. Hu, X., Zhu, L., Fu, C.W., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7454–7462 (2018)

  15. Huang, X., Hua, G., Tumblin, J., Williams, L.: What characterizes a shadow boundary under the sun and sky? In: 2011 International Conference on Computer Vision, pp. 898–905. IEEE (2011)

  16. Junejo, I.N., Foroosh, H.: Estimating geo-temporal location of stationary cameras using shadow trajectories. In: European Conference on Computer Vision, pp. 318–331. Springer (2008)

  17. Karsch, K., Hedau, V., Forsyth, D., Hoiem, D.: Rendering synthetic objects into legacy photographs. ACM Trans. Graph. TOG 30(6), 1–12 (2011)

    Article  Google Scholar 

  18. Khan, S.H., Bennamoun, M., Sohel, F., Togneri, R.: Automatic feature learning for robust shadow detection. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1939–1946. IEEE (2014)

  19. Lalonde, J.F., Efros, A.A., Narasimhan, S.G.: Estimating the natural illumination conditions from a single outdoor image. Int. J. Comput. Vis. 98(2), 123–145 (2012)

    Article  MathSciNet  Google Scholar 

  20. Le, H., Vicente, T.F.Y., Nguyen, V., Hoai, M., Samaras, D.: A+ d net: Training a shadow detector with adversarial shadow attenuation. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 662–678 (2018)

  21. Li, Z., Liu, F., Yang, W., Peng, S., Zhou, J.: A survey of convolutional neural networks: analysis, applications, and prospects. IEEE Trans. Neural Netw. Learn. Syst. (2021). https://doi.org/10.1109/TNNLS.2021.3084827

    Article  Google Scholar 

  22. Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2117–2125 (2017)

  23. Liu, Z., Xiang, Q., Tang, J., Wang, Y., Zhao, P.: Robust salient object detection for RGB images. Vis. Comput. 36(9), 1823–1835 (2020)

    Article  Google Scholar 

  24. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

  25. Mo, J., Zhang, L.: Multi-level deep supervised networks for retinal vessel segmentation. Int. J. Comput. Assist. Radiol. Surg. 12(12), 2181–2193 (2017)

    Article  Google Scholar 

  26. Mohajerani, S., Saeedi, P.: Cpnet: A context preserver convolutional neural network for detecting shadows in single RGB images. In: 2018 IEEE 20th International Workshop on Multimedia Signal Processing (MMSP), pp. 1–5. IEEE (2018)

  27. Nguyen, V., Yago Vicente, T.F., Zhao, M., Hoai, M., Samaras, D.: Shadow detection with conditional generative adversarial networks. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 4510–4518 (2017)

  28. Nielsen, M., Madsen, C.B.: Graph cut based segmentation of soft shadows for seamless removal and augmentation. In: Scandinavian Conference on Image Analysis, pp. 918–927. Springer (2007)

  29. Okabe, T., Sato, I., Sato, Y.: Attached shadow coding: estimating surface normals from shadows under unknown reflectance and lighting conditions. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1693–1700. IEEE (2009)

  30. Panagopoulos, A., Samaras, D., Paragios, N.: Robust shadow and illumination estimation using a mixture model. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 651–658. IEEE (2009)

  31. Panagopoulos, A., Wang, C., Samaras, D., Paragios, N.: Simultaneous cast shadows, illumination and geometry inference using hypergraphs. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 437–449 (2012)

    Article  Google Scholar 

  32. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241. Springer (2015)

  33. Tian, J., Qi, X., Qu, L., Tang, Y.: New spectrum ratio properties and features for shadow detection. Pattern Recognit. 51, 85–96 (2016)

    Article  Google Scholar 

  34. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł, Polosukhin, I.: Attention is all you need. In: Guyon, I., Von Luxburg, U., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30. Curran Associates Inc, Red Hook (2017)

    Google Scholar 

  35. Vicente, T.F.Y., Hoai, M., Samaras, D.: Leave-one-out kernel optimization for shadow detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3388–3396 (2015)

  36. Vicente, T.F.Y., Hou, L., Yu, C.P., Hoai, M., Samaras, D.: Large-scale training of shadow detectors with noisily-annotated shadow examples. In: European Conference on Computer Vision, pp. 816–832. Springer (2016)

  37. Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1788–1797 (2018)

  38. Wang, Y., Wei, X., Ding, L., Tang, X., Zhang, H.: A robust visual tracking method via local feature extraction and saliency detection. Vis. Comput. 36(4), 683–700 (2020)

    Article  Google Scholar 

  39. Wu, W., Zhang, S., Tian, M., Tan, D., Wu, X., Wan, Y.: Learning to detect soft shadow from limited data. Vis. Comput. 38, 1–11 (2021)

    Google Scholar 

  40. Wu, W., Zhou, K., Chen, X.D.: Single image shadow detection via uncertainty analysis and GCN-based refinement strategy. J. Vis. Commun. Image Represent. 82, 103397 (2022)

    Article  Google Scholar 

  41. Wu, W., Zhou, K., Chen, X.D., Yong, J.H.: Light-weight shadow detection via GCN-based annotation strategy and knowledge distillation. Comput. Vis. Image Underst. 216, 103341 (2022)

    Article  Google Scholar 

  42. Yao, Q., Yang, H., Han, B., Niu, G., Kwok, J.T.Y.: Searching to exploit memorization effect in learning with noisy labels. In: International Conference on Machine Learning, pp. 10789–10798. PMLR (2020)

  43. Zhang, J., Yu, X., Li, A., Song, P., Liu, B., Dai, Y.: Weakly-supervised salient object detection via scribble annotations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12546–12555 (2020)

  44. Zheng, Q., Qiao, X., Cao, Y., Lau, R.W.: Distraction-aware shadow detection. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5167–5176 (2019)

  45. Zhou, K., Wu, W., Shao, Y.L., Fang, J.L., Wang, X.Q., Wei, D.: Shadow detection via multi-scale feature fusion and unsupervised domain adaptation. J. Vis. Commun. Image Represent. 88, 103596 (2022)

    Article  Google Scholar 

  46. Zhu, J., Samuel, K.G., Masood, S.Z., Tappen, M.F.: Learning to recognize shadows in monochromatic natural images. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 223–230. IEEE (2010)

  47. Zhu, L., Deng, Z., Hu, X., Fu, C.W., Xu, X., Qin, J., Heng, P.A.: Bidirectional feature pyramid network with recurrent attention residual modules for shadow detection. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 121–136 (2018)

Download references

Acknowledgements

This work is supported by the Natural Science Foundation of Xinjiang Uygur Autonomous Region (Nos. 2019D01C062, 2019D01C041, 2019D01C205, 2020D01C028), the National Natural Science Foundation of China (No. 12061071), the Higher Education of Xinjiang Uygur Autonomous Region (XJEDU2019Y006, XJEDU2020Y003), Tianshan Innovation Team Plan Project of Xinjiang Uygur Autonomous Region under Grant (No. 202101642), and Sichuan Regional Innovation Cooperation Project (No. 2020YFQ0018), the National Social Science Foundation in China (No. 20XGL029).

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Xinjiang University, Urumqi, 830046, China

    Xian-Tao Wu & Lin-Lin Zhang

  2. School of Computer Science and Technology, Hangzhou Dianzi University, Hangzhou, 310018, China

    Wen Wu

  3. Department of Computer Science and Engineering, Shaoxing University, Shaoxing, 312000, China

    Yi Wan

Authors
  1. Xian-Tao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wen Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lin-Lin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yi Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xian-Tao Wu or Lin-Lin Zhang.

Ethics declarations

Conflict of interest

The authors declare that they all have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, XT., Wu, W., Zhang, LL. et al. Don’t worry about noisy labels in soft shadow detection. Vis Comput 39, 6297–6308 (2023). https://doi.org/10.1007/s00371-022-02730-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02730-9

Keywords

Search

Navigation