Skip to main content

Advertisement

Springer Nature Link
Log in

TSPLNet: a three-stage progressive lightweight network for shadow removal

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Shadow removal is a significant yet challenging task in computer vision. Most existing methods for shadow removal rely on “black-box” models, which lack transparency and fail to fully integrate shadow removal theory. Moreover, their large number of parameters renders them unsuitable for scenarios with limited computational resources. This study introduces TSPLNet, a lightweight and progressive network for shadow removal, combining a physics-based model with a data-driven approach. First, we incorporate shadow removal principles into image restoration theory to redefine the shadow removal process. Based on this new computational framework, we design a three-stage progressive deep learning network, where each stage iteratively refines shadow reconstruction. To improve the model’s ability to extract and reconstruct shadow features, we introduce key components such as a deformable residual block, a shadow attention adjuster, and a shadow interaction attention module, along with a redesigned loss function. Furthermore, we apply depthwise separable convolution, reducing the model’s parameters to 36% of the original. Experiments on the ISTD and SRD datasets, as well as crop leaf images demonstrate that our method significantly outperforms other advanced methods in terms of parameter count, quantitative metrics, and visual quality, highlighting its effectiveness in image shadow removal.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data availability

No datasets were generated or analysed during the current study.

References

  1. Nadimi, S., Bhanu, B.: Physical models for moving shadow and object detection in video. IEEE Trans. Pattern Anal. Mach. Intell. 26, 1079–1087 (2004)

    Article  MATH  Google Scholar 

  2. Sultana, M., Mahmood, A., Jung, S.K.: Unsupervised moving object detection in complex scenes using adversarial regularizations. IEEE Trans. Multimedia. 23, 2005–2018 (2021)

    Article  MATH  Google Scholar 

  3. Shao, Z., Pu, Y., Zhou, J., Wen, B., Zhang, Y.: Hyper RPCA: Joint maximum correntropy criterion and laplacian scale mixture modeling on-the-fly for moving object detection. IEEE Trans. Multimedia. 25, 112–125 (2020). 

    Article  MATH  Google Scholar 

  4. Saravanakumar, S., Vadivel, A., Saneem Ahmed, C.G.: Multiple human object tracking using background subtraction and shadow removal techniques. Proc. Int. Conf. Signal Image Process. 1, 79–84 (2010)

    MATH  Google Scholar 

  5. Liu, C., Liu, P., Zhao, W., Tang, X.: Robust tracking and redetection: Collaboratively modeling the target and its context. IEEE Trans. Multimedia. 20, 889–902 (2018)

    Article  MATH  Google Scholar 

  6. Liang, N., Wu, G., Kang, W., Wang, Z., Feng, D.D.: Real-time long-term tracking with prediction-detection-correction. IEEE Trans. Multimedia. 20, 2289–2302 (2018)

    Article  MATH  Google Scholar 

  7. Zhang, T., et al.: Decoupled spatial neural attention for weakly supervised semantic segmentation. IEEE Trans. Multimedia. 21, 2930–2941 (2019)

    Article  MATH  Google Scholar 

  8. Yu, C., et al.: BiSeNet: Bilateral segmentation network for real-time semantic segmentation. Proc. Eur. Conf. Comput. Vis. 2018, 334–349 (2018)

    MATH  Google Scholar 

  9. Kang, B., Lee, Y., Nguyen, T.Q.: Depth-adaptive deep neural network for semantic segmentation. IEEE Trans. Multimedia. 20, 2478–2490 (2018)

    Article  MATH  Google Scholar 

  10. Weiss, Y.: Deriving intrinsic images from image sequences. Proc. Eighth IEEE Int. Conf. Comput. Vis. 2, 68–75 (2001). 

    Article  MATH  Google Scholar 

  11. Finlayson, G.D., Drew, M.S., Lu, C.: Entropy minimization for shadow removal. Int. J. Comput. Vis. 85, 35–57 (2009)

    Article  MATH  Google Scholar 

  12. Finlayson, G.D., Hordley, S.D., Lu, C., Drew, M.S.: On the removal of shadows from images. IEEE Trans. Pattern Anal. Mach. Intell. 28, 59–68 (2005)

    Article  MATH  Google Scholar 

  13. Guo, R., Dai, Q., Hoiem, D.: Single-image shadow detection and removal using paired regions. IEEE Conf. Comput. Vis. Pattern Recogn. 1, 2033–2040 (2011)

    MATH  Google Scholar 

  14. Huang, X., Hua, G., Tumblin, J., Williams, L.: What characterizes a shadow boundary under the sun and sky? Proc. IEEE/CVF Int. Conf. Comput. Vis. 2, 898–905 (2011)

    MATH  Google Scholar 

  15. Mohan, A., Tumblin, J., Choudhury, P.: Editing soft shadows in a digital photograph. IEEE Comput. Graph Appl. 27, 23–31 (2007)

    Article  MATH  Google Scholar 

  16. Hu, X., Fu, C.W., Zhu, L., Qin, J., Heng, P.A.: Direction-aware spatial context features for shadow detection and removal. IEEE Trans. Pattern Anal. Mach. Intell. 42, 2795–2808 (2019)

    Article  MATH  Google Scholar 

  17. Qu, L., Tian, J., He, S., Tang, Y., Lau, R.W.H.: Deshadownet: A multi-context embedding deep network for shadow removal. IEEE Conf. Comput. Vis. Pattern Recogn. 1, 4067–4075 (2017)

    MATH  Google Scholar 

  18. Wang, J., Li, X., Yang, J.: Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. Proc. IEEE Conf. Comput. Vis. Pattern Recogn. 1, 1788–1797 (2018)

    MATH  Google Scholar 

  19. Jin, Y., Sharma, A., Tan, R.T.: Dc-shadownet: Single-image hard and soft shadow removal using unsupervised domain-classifier guided network. Proc. IEEE/CVF Int. Conf. Comput. Vis. 1, 5007–5016 (2021)

    MATH  Google Scholar 

  20. Le, H., Samaras, D.: Shadow removal via shadow image decomposition.Proc. IEEE/CVF Int. Conf. Comput. Vis. 1, 8578-8587 (2019)

  21. Ding, B., Long, C., Zhang, L., Xiao, C.: Argan: Attentive recurrent generative adversarial network for shadow detection and removal. Proc. IEEE/CVF Int. Conf. Comput. Vis. 1,10213-10222(2019).https://openaccess.thecvf.com/content_ICCV_2019/html/Le_Shadow_Removal_via_Shadow_Image_Decomposition_ICCV_2019_paper.html

  22. Chen, Z., Long, C., Zhang, L., Xiao, C.: Canet: A context-aware network for shadow removal. Proc. IEEE/CVF Int. Conf. Comput. Vis. 1, 4723–4732 (2021). https://openaccess.thecvf.com/content_ICCV_2019/html/Ding_ARGAN_Attentive_Recurrent_Generative_Adversarial_Network_for_Shadow_Detection_and_ICCV_2019_paper.html

    MATH  Google Scholar 

  23. Xu, Z., Chen, X.: A two-stage progressive shadow removal network. Appl. Intell. 53, 25296–25309 (2023)

    Article  Google Scholar 

  24. Zhu, Y., et al.: Bijective mapping network for shadow removal. Proc IEEE/CVF Conf. Comput. Vis. Pattern Recogn 1, 5617–5626 (2022)

    MATH  Google Scholar 

  25. Mou, C., Wang, Q., Zhang, J.: Deep generalized unfolding networks for image restoration. Proc IEEE/CVF Conf. Comput. Vis. Pattern Recogn. 1, 17399–17410 (2022)

    Google Scholar 

  26. Buades, A., Coll, B., Morel, J.-M.: A nonlocal algorithm for image denoising. Proc IEEE/CVF Conf. Comput. Vis. Pattern Recogn 1, 60–65 (2005)

    MATH  Google Scholar 

  27. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16, 2080–2095 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  28. Porter, T., Duff, T.: Compositing digital images. Comput. Graph Interact. Tech. 1, 253–259 (1984)

    MATH  Google Scholar 

  29. Dai, J., et al.: Deformable convolutional networks. Proc. IEEE Int. Conf. Comput. Vis. 1, 764–773 (2017)

    MATH  Google Scholar 

  30. Guo, L., Huang, S., Liu, D., et al.: Shadowformer: Global context helps image shadow removal. arXiv Preprint ArXiv 1, 230201650 (2023)

    Google Scholar 

  31. Sifre, L., Mallat, S.: Rigid-motion scattering for texture classification. Arxiv Preprint ArXiv 1, 14031687 (2014)

    MATH  Google Scholar 

  32. Seif, G., Androutsos, D.: Edge-based loss function for single image super-resolution. IEEE Int. Conf. Acoust. Speech Signal Process. 2018, 1468–1472 (2018)

    MATH  Google Scholar 

  33. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super resolution. In: Comput. Vis.–ECCV: Proc. Part II: 14th European Conference, Amsterdam, The Netherlands, Oct. 11–14, 2016, vol. 14, pp. 694–711. (2016)

  34. Cun, X., Pun, C.M., Shi, C.: Towards ghost-free shadow removal via dual hierarchical aggregation network and shadow matting GAN. AAAI. 34, 10680–10687 (2020)

    Article  MATH  Google Scholar 

  35. Fu, L., et al.: Auto-exposure fusion for single-image shadow removal. Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recogn.1,10566–10575(2021)

    Google Scholar 

  36. Zhu, Y., et al.: Efficient model-driven network for shadow removal. AAAI 36, 3635–3643 (2022b)

    Article  MATH  Google Scholar 

  37. Wang, Y., Zhou, W., Feng, H., Li, L., Li, H.: Progressive recurrent network for shadow removal. Comput. Vis. Image Underst. 238, 103861 (2024)

    Article  MATH  Google Scholar 

  38. Zamir, S.W., et al.: Multi-stage progressive image restoration. Proc. IEEE/CVF Conf. Comput. Vis. Pattern Recogn. 1, 14816–14826 (2021)

    Google Scholar 

Download references

Acknowledgements

This work is partly supported by the National Natural Science Foundation of China through grant 72071163, the Youth Foundation of Inner Mongolia Natural Science Foundation through grant 2024QN06017, the Inner Mongolia Natural Science Foundation through grant 2021MS06007, and the Basic Scientific Research Business Fee Project for Universities in Inner Mongolia through grants 0406082215, 2023XKJX019 and 2023XKJX024.

Author information

Authors and Affiliations

  1. School of Transportation and Logistics, Southwest Jiaotong University, Chengdu, 611730, China

    Weijian Hu & Ke Han

  2. School of Digital and Intelligent Industry, Inner Mongolia University of Science and Technology, Baotou, 014000, China

    Weijian Hu, Yinyin Xu, Lingfang Li & Jiang Wang

  3. School of Information Science and Technology, Southwest Jiaotong University, Chengdu, 611730, China

    Lingfang Li

Authors
  1. Weijian Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yinyin Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Ke Han
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Lingfang Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Jiang Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Hu is responsible for designing the overall plan, writing, and revising the paper. Xu is responsible for the plan’s implementation. Han is in charge of creating the model’s theoretical methods. Li is responsible for revising and improving the paper. Wang is responsible for other matters.

Corresponding authors

Correspondence to Weijian Hu or Ke Han.

Ethics declarations

Conflict of interest

The authors declare that they have no conflicts of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, W., Xu, Y., Han, K. et al. TSPLNet: a three-stage progressive lightweight network for shadow removal. Multimedia Systems 31, 29 (2025). https://doi.org/10.1007/s00530-024-01607-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00530-024-01607-2