DMAED: Dynamic Matte Aware Encoder-Decoder for Shadow Removal
Authors: 
Yadong Zhou, Zhenfeng ZhuAuthors Info & Claims
Yadong Zhou, Zhenfeng ZhuAuthors Info & ClaimsPages 251 - 256
Abstract
The removal of shadows in images has significant implications for research in semantic segmentation and object recognition. Although methods based on deep learning models have made important progress in shadow removal, there is still room for improvement in the simultaneous treatment of soft and hard shadows. To address these challenges and improve the robustness of the algorithms, this study introduces a dynamic matte-attention mechanism into the encoder-decoder network and proposes the DMAED (Dynamic Matte Aware Encoder-Decoder) model. The DMAED model first preprocesses the input data using a matte mask. Then, through iterative processes and updates of tokens and dynamic masks, it facilitates the migration of features from non-shadow to shadow regions, gradually eliminating shadow artifacts. Finally, a cascaded refinement decoder further rectifies shadowed images. Experimental results on benchmark datasets such as ISTD, ISTD+, and SRD demonstrate the significant effectiveness of the DMAED model in handling both hard and soft shadow scenarios.
References
[1]
Sahoo, Subhaluxmi, and Pradipta Kumar Nanda. "Adaptive Feature Fusion and Spatio-Temporal Background Modeling in KDE Framework for Object Detection and Shadow Removal." IEEE Transactions on Circuits and Systems for Video Technology 32.3 2022.
[2]
Pang Yu, Xia Ruiyang, Lin Jin chao, Li Guoquan, Huang Zhengwen, and Wen Lingyun. Shadow Detection Method Assisted by Data Set Expansion and Shadow Image Classification 2022.
[3]
Wieland, Marc, Yu Li, and Sandro Martinis. "Multi-sensor Cloud and Cloud Shadow Segmentation with a Convolutional Neural Network." Remote Sensing of Environment 230 2019: 111203.
[4]
Barrow, H G, and J. M. Tenenbaum. "Recovering Intrinsic Scene Characteristics from Images." computer vision systems 1978.
[5]
G. Finlayson, S. Hordley, C. Lu, and M. Drew, "On the removal of shadows from images," IEEE Trans. Pattern Anal. Mach. Intell, vol. 28, no. 1, pp. 59–68, Jan. 2006.
[6]
Guo, Ruiqi, Q. Dai, and D Hoiem." Paired Regions for Shadow Detection and Removal." IEEE Transactions on Pattern Analysis & Machine Intelligence 35.12 2013.
[7]
Zali, Siti-Aisyah, Shahbe Mat-Desa, Zarina CheEmbi, and Wan-Noorshahida Mohd-Isa. "Post-Processing for Shadow Detection in Drone-Acquired Images Using U-NET." Future Internet 14.8 2022: 231.
[8]
Fan, Zhun, Huibiao Lin, Chong Li, Jian Su, Salvatore Bruno, and Giuseppe Loprencipe. "Use of Parallel ResNet for High-Performance Pavement Crack Detection and Measurement." Sustainability (Basel, Switzerland) 14.3 2022: 1825.
[9]
Qu, L, Tian, J, He, S, Tang, Y, & Lau, R. W. H 2017. DeshadowNet: A Multi-context Embedding Deep Network for Shadow Removal. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE.
[10]
Jin, Y, Sharma, A, & Tan, R T. 2021. DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network. International Conference on Computer Vision.
[11]
Chen, Zipei, Chengjiang Long, Ling Zhang, and Chunxia Xiao. "CANet: A Context-Aware Network for Shadow Removal." 2021 IEEE/CVF International Conference on Computer Vision (ICCV) 2021: 4723-732.
[12]
Guo, Lanqing, Siyu Huang, Ding Liu, Hao Cheng, and Bihan Wen. "Shadow- Former: Global Context Helps Image Shadow Removal." 2023.
[13]
Land, E. H. 1978. The Retinex theory of color vision. Scientific American, 237(6), 108-128.
[14]
T. Porter and T. Duff, "Compositing digital images," Proc. ACM SIGGRAPH, Comput. Graph. Forum, vol. 18, no. 3, pp. 253–259, Jan. 1984.
[15]
H. Le and D. Samaras, "Physics-Based Shadow Image Decomposition for Shadow Removal," in IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 44, no. 12, pp. 9088-9101, 1 Dec. 2022.
[16]
Qu, L.; Tian, J.; He, S.; Tang, Y.; and Lau, R. W. 2017.Deshadownet: A multi-context embedding deep network for shadow removal. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 4067–4075.
[17]
Wang, J.; Li, X.; and Yang, J. 2018. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1788–1797.
[18]
Le, H.; and Samaras, D. 2019. Shadow removal via shadow image decomposition. In Proc. IEEE Int'l Conf. Computer Vision, 8578–8587.
[19]
Cun, X.; Pun, C.-M.; and Shi, C. 2020. Towards Ghost-Free Shadow Removal via Dual Hierarchical Aggregation Network and Shadow Matting GAN. In Proc. AAAI Conf. on Artificial Intelligence, 10680–10687.
[20]
Zheng, Chaobing, "Single Image Brightening via Multi-Scale Exposure Fusion with Hybrid Learning." Institute of Electrical and Electronics Engineers (IEEE) 4 2021.
[21]
Zhu, Y.; Xiao, Z.; Fang, Y.; Fu, X.; Xiong, Z.; and Zha, Z.-J. 2022. Efficient Model-Driven Network for Shadow Removal.
[22]
Xu Y, Lin M, Yang H, Shadow-Aware Dynamic Convolution for Shadow Removal[J]. 2022.
[23]
Y.-Y. Chuang, D. B. Goldman, B. Curless, D. H. Salesin "Shadow matting and compositing," ACM Trans. Graph., vol. 22, no. 3, pp. 494–500, Jul. 2003.
[24]
A. Levin, D. Lischinski, and Y. Weiss, "A closed-form solution to natural image matting," IEEE Trans. Pattern Anal. Mach. Intell, vol. 30, no. 2, pp. 228–242, Feb. 2008.
Index Terms
- DMAED: Dynamic Matte Aware Encoder-Decoder for Shadow Removal
Recommendations
Precomputed shadow fields for dynamic scenes
SIGGRAPH '05: ACM SIGGRAPH 2005 PapersWe present a soft shadow technique for dynamic scenes with moving objects under the combined illumination of moving local light sources and dynamic environment maps. The main idea of our technique is to precompute for each scene entity a shadow field ...
Precomputed shadow fields for dynamic scenes
We present a soft shadow technique for dynamic scenes with moving objects under the combined illumination of moving local light sources and dynamic environment maps. The main idea of our technique is to precompute for each scene entity a shadow field ...
Shadow-aware dynamic convolution for shadow removal
AbstractWith a wide range of shadows in many collected images, shadow removal has aroused increasing attention since uncontaminated images are of vital importance for many computer vision tasks. Current methods consider the same convolution operations ...
Highlights- We propose a plug-and-play shadow-aware dynamic convolution module to handle the shadow regions and non-shadow regions separately.
- We design a specific intra-convolution distillation loss to enhance the shadow removal performance based ...
Comments
Information & Contributors
Information
Published In
March 2024
676 pages
ISBN:9798400718212
DOI:10.1145/3672919
Copyright © 2024 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 24 July 2024
Check for updates
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
CSAIDE 2024
CSAIDE 2024: 2024 3rd International Conference on Cyber Security, Artificial Intelligence and Digital Economy
March 1 - 3, 2024
Nanjing, China
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 10Total Downloads
- Downloads (Last 12 months)10
- Downloads (Last 6 weeks)2
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format