skip to main content
10.1145/3672919.3672966acmotherconferencesArticle/Chapter ViewAbstractPublication PagescsaideConference Proceedingsconference-collections
research-article

DMAED: Dynamic Matte Aware Encoder-Decoder for Shadow Removal

Published: 24 July 2024 Publication History

Abstract

The removal of shadows in images has significant implications for research in semantic segmentation and object recognition. Although methods based on deep learning models have made important progress in shadow removal, there is still room for improvement in the simultaneous treatment of soft and hard shadows. To address these challenges and improve the robustness of the algorithms, this study introduces a dynamic matte-attention mechanism into the encoder-decoder network and proposes the DMAED (Dynamic Matte Aware Encoder-Decoder) model. The DMAED model first preprocesses the input data using a matte mask. Then, through iterative processes and updates of tokens and dynamic masks, it facilitates the migration of features from non-shadow to shadow regions, gradually eliminating shadow artifacts. Finally, a cascaded refinement decoder further rectifies shadowed images. Experimental results on benchmark datasets such as ISTD, ISTD+, and SRD demonstrate the significant effectiveness of the DMAED model in handling both hard and soft shadow scenarios.

References

[1]
Sahoo, Subhaluxmi, and Pradipta Kumar Nanda. "Adaptive Feature Fusion and Spatio-Temporal Background Modeling in KDE Framework for Object Detection and Shadow Removal." IEEE Transactions on Circuits and Systems for Video Technology 32.3 2022.
[2]
Pang Yu, Xia Ruiyang, Lin Jin chao, Li Guoquan, Huang Zhengwen, and Wen Lingyun. Shadow Detection Method Assisted by Data Set Expansion and Shadow Image Classification 2022.
[3]
Wieland, Marc, Yu Li, and Sandro Martinis. "Multi-sensor Cloud and Cloud Shadow Segmentation with a Convolutional Neural Network." Remote Sensing of Environment 230 2019: 111203.
[4]
Barrow, H G, and J. M. Tenenbaum. "Recovering Intrinsic Scene Characteristics from Images." computer vision systems 1978.
[5]
G. Finlayson, S. Hordley, C. Lu, and M. Drew, "On the removal of shadows from images," IEEE Trans. Pattern Anal. Mach. Intell, vol. 28, no. 1, pp. 59–68, Jan. 2006.
[6]
Guo, Ruiqi, Q. Dai, and D Hoiem." Paired Regions for Shadow Detection and Removal." IEEE Transactions on Pattern Analysis & Machine Intelligence 35.12 2013.
[7]
Zali, Siti-Aisyah, Shahbe Mat-Desa, Zarina CheEmbi, and Wan-Noorshahida Mohd-Isa. "Post-Processing for Shadow Detection in Drone-Acquired Images Using U-NET." Future Internet 14.8 2022: 231.
[8]
Fan, Zhun, Huibiao Lin, Chong Li, Jian Su, Salvatore Bruno, and Giuseppe Loprencipe. "Use of Parallel ResNet for High-Performance Pavement Crack Detection and Measurement." Sustainability (Basel, Switzerland) 14.3 2022: 1825.
[9]
Qu, L, Tian, J, He, S, Tang, Y, & Lau, R. W. H 2017. DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
[10]
Jin, Y, Sharma, A, & Tan, R T. 2021. DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network. International Conference on Computer Vision.
[11]
Chen, Zipei, Chengjiang Long, Ling Zhang, and Chunxia Xiao. "CANet: A Context-Aware Network for Shadow Removal." 2021 IEEE/CVF International Conference on Computer Vision (ICCV) 2021: 4723-732.
[12]
Guo, Lanqing, Siyu Huang, Ding Liu, Hao Cheng, and Bihan Wen. "Shadow- Former: Global Context Helps Image Shadow Removal." 2023.
[13]
Land, E. H. 1978. The Retinex theory of color vision. Scientific American, 237(6), 108-128.
[14]
T. Porter and T. Duff, "Compositing digital images," Proc. ACM SIGGRAPH, Comput. Graph. Forum, vol. 18, no. 3, pp. 253–259, Jan. 1984.
[15]
H. Le and D. Samaras, "Physics-Based Shadow Image Decomposition for Shadow Removal," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 9088-9101, 1 Dec. 2022.
[16]
Qu, L.; Tian, J.; He, S.; Tang, Y.; and Lau, R. W. 2017.Deshadownet: A multi-context embedding deep network for shadow removal. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 4067–4075.
[17]
Wang, J.; Li, X.; and Yang, J. 2018. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1788–1797.
[18]
Le, H.; and Samaras, D. 2019. Shadow removal via shadow image decomposition. In Proc. IEEE Int'l Conf. Computer Vision, 8578–8587.
[19]
Cun, X.; Pun, C.-M.; and Shi, C. 2020. Towards Ghost-Free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN. In Proc. AAAI Conf. on Artificial Intelligence, 10680–10687.
[20]
Zheng, Chaobing, "Single Image Brightening via Multi-Scale Exposure Fusion with Hybrid Learning." Institute of Electrical and Electronics Engineers (IEEE) 4 2021.
[21]
Zhu, Y.; Xiao, Z.; Fang, Y.; Fu, X.; Xiong, Z.; and Zha, Z.-J. 2022. Efficient Model-Driven Network for Shadow Removal.
[22]
Xu Y, Lin M, Yang H, Shadow-Aware Dynamic Convolution for Shadow Removal[J]. 2022.
[23]
Y.-Y. Chuang, D. B. Goldman, B. Curless, D. H. Salesin "Shadow matting and compositing," ACM Trans. Graph., vol. 22, no. 3, pp. 494–500, Jul. 2003.
[24]
A. Levin, D. Lischinski, and Y. Weiss, "A closed-form solution to natural image matting," IEEE Trans. Pattern Anal. Mach. Intell, vol. 30, no. 2, pp. 228–242, Feb. 2008.

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
CSAIDE '24: Proceedings of the 2024 3rd International Conference on Cyber Security, Artificial Intelligence and Digital Economy
March 2024
676 pages
ISBN:9798400718212
DOI:10.1145/3672919
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 July 2024

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

CSAIDE 2024

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 10
    Total Downloads
  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)2
Reflects downloads up to 01 Mar 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media