Xiaoyue Jiang
ABSTRACT
Shadows actually play an important role in image understanding. But even for the same object, the intensity and shape of the shadows can vary with the environment. Thus it is a quite changeling issue to detect and remove shadows from images. Recent studies have been trying to solve these two tasks independently, but they are closely related to each other actually. Therefore, we propose a multi-task adversarial generative networks (mtGAN) that can detect and remove shadows simultaneously. For the proposed mtGAN, the cross-stitch unit is applied to learn the optimal ways to share features between multi-tasks, which is not set empirically as usual. Also, the combination weight of multi-task loss functions are trained according to the uncertainty distribution of each task. Based on these multi-task learning strategies, the proposed mtGAN can achieve shadow detection and removal tasks better than existing methods. In experiments, the effectiveness of the proposed mtGAN is shown.
- J. T. Barron and J. Malik. 2015. Shape, illumination, and reflectance from shading. IEEE Trans. on PAMI 37, 8 (2015), 1670–1687.Google Scholar
Digital Library
- C. Benedek and T. Szirányi. 2008. Bayesian foreground and shadow detection in uncertain frame rate surveillance videos. IEEE Trans. on IP 17, 4 (2008), 608–621.Google ScholarDigital Library
- R. Cipolla, Y. Gal, and A. Kendall. 2018. Multi-task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics. CVPR (2018), 7482–7491.Google Scholar
- B. Ding, C. Long, L. Zhang, and C. Xiao. 2019. ARGAN: Attentive Recurrent Generative Adversarial Network for Shadow Detection and Removal.ICCV (2019).Google Scholar
- G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew. 2006. On the removal of shadows from images. IEEE Trans. on PAMI 28, 1 (2006), 59–68.Google ScholarDigital Library
- G. D. Finlayson, S. D. Hordley, C. Lu, and M. S. Drew. 2006. On the removal of shadows from images. IEEE Trans. PAMI 28, 1 (2006), 59–68.Google ScholarDigital Library
- H. Gong and D.P. Cosker. 2014. Interactive shadow removal and ground truth for variable scene categories. In BMVC.Google Scholar
- M. Gryka, M. Terry, and G. J. Brostow. 2015. Learning to remove soft shadows. ACM Trans. on Graphics (TOG) 34, 5 (2015), 153.Google ScholarDigital Library
- R. Guo, Q. Dai, and D. Hoiem. 2011. Single-image shadow detection and removal using paired regions. In CVPR. 2033–2040.Google Scholar
- K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep Residual Learning for Image Recognition. In CVPR. 770–778.Google Scholar
- Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167(2015).Google Scholar
- X. Jiang, A. J. Schofield, and J. L. Wyatt. 2011. Shadow Detection based on Colour Segmentation and Estimated Illumination.BMVC (2011), 1–11.Google Scholar
- Justin Johnson, Alexandre Alahi, and Li Fei-Fei. 2016. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision. Springer, 694–711.Google ScholarCross Ref
- S. H. Khan, M. Bennamoun, F. Sohel, and R. Togneri. 2016. Automatic shadow detection and removal from a single image. IEEE Trans. on PAMI3(2016), 431–446.Google Scholar
- J.-F. Lalonde, A. A. Efros, and S. G. Narasimhan. 2010. Detecting ground shadows in outdoor consumer photographs. In ECCV. Springer, 322–335.Google Scholar
- H. Le and D. Samaras. 2019. Shadow Removal via Shadow Image Decomposition.ICCV (2019).Google Scholar
- M. Long and J. Wang. 2015. Learning Multiple Tasks with Deep Relationship Networks.arXiv: Learning (2015).Google Scholar
- I. Misra, A. Shrivastava, A. Gupta, and M. Hebert. 2016. Cross-Stitch Networks for Multi-task Learning. CVPR (2016), 3994–4003.Google Scholar
- M. Qi, J. Dai, Q. Zhang, and J. Kong. 2014. Cascaded cast shadow detection method in surveillance scenes. Optik-International Journal for Light and Electron Optics 125, 3(2014), 1396–1400.Google ScholarCross Ref
- L. Qu, J. Tian, S. He, Y. Tang, and R. Lau. 2017. Deshadownet: A multi-context embedding deep network for shadow removal. In CVPR. 3.Google Scholar
- R. Ranjan, V. M. Patel, and R. Chellappa. 2019. HyperFace: A Deep Multi-Task Learning Framework for Face Detection, Landmark Localization, Pose Estimation, and Gender Recognition. IEEE Trans. on PAMI 41, 1 (2019), 121–135.Google ScholarDigital Library
- L. Shen, T. W. Chua, and K. Leman. 2015. Shadow optimization from structured deep edge detection. In CVPR. IEEE, 2067–2074.Google Scholar
- E. Vazquez, R. Baldrich, J. V. De Weijer, and M. Vanrell. 2011. Describing Reflectances for Color Segmentation Robust to Shadows, Highlights, and Textures. IEEE Trans. on PAMI 33, 5 (2011), 917–930.Google ScholarDigital Library
- J. Wang, X. Li, and J. Yang. 2018. Stacked conditional generative adversarial networks for jointly learning shadow detection and shadow removal. In CVPR. 1788–1797.Google Scholar
- Q. Yang, K.-H. Tan, and N. Ahuja. 2012. Shadow removal using bilateral filtering. IEEE Trans. on IP 21, 10 (2012), 4361–4368.Google ScholarDigital Library
- Q. Zheng, X. Qiao, Y. Cao, and R. W. H. Lau. 2019. Distraction-Aware Shadow Detection. CVPR (2019), 5167–5176.Google Scholar
Index Terms
- Deep Multi-task Learning for Shadow Detection and Removal
Recommendations
Shadow Detection and Removal Based on Multi-task Generative Adversarial Networks
Image and GraphicsAbstractThe existence of shadows is difficult to avoid in images. Also, it will affect object recognition and image understanding. But on the other hand, shadow can provide information about the light source and object shape. Therefore, accurate shadow ...
Simple shadow removal using shadow depth map and illumination-invariant feature
AbstractShadows included in images provide useful information for visual scene analysis, but are also factors that negatively affect digital image analysis. Therefore, shadow detection and removal must be considered essential in the preprocessing of the ...
Shadow Remover: Image Shadow Removal Based on Illumination Recovering Optimization
In this paper, we present a novel shadow removal system for single natural images as well as color aerial images using an illumination recovering optimization method. We first adaptively decompose the input image into overlapped patches according to the ...
Comments