Yirui Zeng
ABSTRACT
Image de-raining aims to eliminate rain streaks captured by outdoor equipment such as video surveillance, remote sensor and automatic pilot. Recently, a de-raining method called non-locally enhanced encoder-decoder network (NLEDN) has achieved reliability performance. Nevertheless, it is very time consuming (2.2571s per image) and takes up memory so that it cannot be applied to mobile devices to process image in real-time. To solve this problem, we design a lightweight channel-spatial attention network that is 55 times faster (41ms per image) and memory saving. The most advanced performances are achieved in most de-raining data sets. More specifically, we design a channel-spatial attention dense block (CSADB). The channel attention operation will be carried out together with the spatial attention. Our experiments demonstrate that the network can learn more effective features by this way. In order to make our proposed method more lightweight, the depthwise convolutions are adapted in each block to reduce parameters. We conduct experiments on four public synthetic datasets to demonstrate the effectiveness of our proposed method, which achieve excellent performance. And the real-world de-raining results are also tacked into comparison. Moreover, an additional experiment demonstrates that our method also works well on face hallucination task. The relevant code and trained models will be available in GitHub soon.
- Yilei Chen and Chiou Ting Hsu. 2013. A Generalized Low-Rank Appearance Model for Spatio-temporally Correlated Rain Streaks. In IEEE International Conference on Computer Vision. 1968--1975.Google Scholar
Digital Library
- Li-Wei Kang, Chia-Wen Lin and Yu-Hsiang Fu. 2012. Automatic single-image-based rain streaks removal via image decomposition. IEEE Transactions on Image Processing, vol. 21. 1742--1755.Google ScholarDigital Library
- Yu Li, Robby T. Tan, Xiaojie Guo, Jiangbo Lu and Michael S. Brown. 2016. Rain streak removal using layer priors. IEEE Conference on Computer Vision and Pattern Recognition. 2736--2744.Google Scholar
- Wenhan Yang, Robby T. Tan, Jiashi Feng, Jiaying Liu, Zongming Guo, and Shuicheng Yan. 2017. Deep Joint Rain Detection and Removal from a Single Image. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- He Zhang and Vishal M. Patel. 2018. Density-aware Single Image De-raining using a Multi-stream Dense Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google Scholar
- Xueyang Fu, Jiabin Huang, Delu Zeng, Yue Huang, Xinghao Ding, and John Paisley. 2017. Removing Rain from Single Images via a Deep Detail Network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1715--1723.Google ScholarCross Ref
- Guanbin Li, Xiang He, Wei Zhang, Huiyou Chang, Le Dong and Liang Lin. 2018. Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining. In ACM International Conference on Multimedia.Google Scholar
- Xiaolong Wang, Ross Girshick, Abhinav Gupta, and Kaiming He. 2018. Non-local Neural Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.Google ScholarCross Ref
- Andrew G. Howard, Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto and Hartwig Adam. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications. arXiv:Computer Vision and Pattern Recognition.Google Scholar
- Jie Hu, Li Shen, Samuel Albanie, Gang Sun and Enhua Wu. 2017. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence.Google Scholar
- Quan Huynh-Thu and Mohammed Ghanbari. 2008. Scope of validity of PSNR in image/video quality assessment. Electronices Letters 44, 13 (2008), 800--8001.Google ScholarCross Ref
- Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. 2004. Image quality assessment: from error visibility to structural similarity. IEEE transactions on image processing 13, 4(2004), 600--612.Google Scholar
- Tai-Xiang Jiang, Ting-Zhu Huang, Xi-Le Zhao, Liang-Jian Deng, Yao Wang. 2018. FastDeRain: A Novel Video Rain Streak Removal Method Using Directional Gradient Priors. IEEE Transactions on Image Processing. 1--1.Google Scholar
- Michael Blot, Matthieu Cord and Nicolas Thome. 2016. Maxmin convolutional neural networks for image classification. In IEEE International Conference on Image Processing (ICIP).Google ScholarCross Ref
- Bing Xu, Naiyan Wang, Tianqi Chen and Mu Li. 2015. Empirical Evaluation of Rectified Activations in Convolutional Network. Computer Science.Google Scholar
- David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. 2001. A database of human segmented natural images and its application to evaluating semantation algorithms and meaduring ecological statistics. In Proceedings of the IEEE Internatioanl Conference on Computer Vision, vol.2. 416--423.Google Scholar
- Chih-Yuan Yang, Sifei Liu and Ming-Hsuan Yang. 2013. Structured Face Hallucination. IEEE Conference on Computer Vision and Pattern Recognition. 1099--1106.Google Scholar
- Erjin Zhou, Haoqiang Fan, Zhimin Cao, Yuning Jiang and Qi Yin. 2015. Learning face hallucination in the wild. In AAAI Conference on Artificial Intelligence. 3871--3877.Google Scholar
- Xiang Ma, Junping Zhang and Chun Qi. 2010. Hallucinating face by position-patch. Pattern Recognition. 2224--2236.Google Scholar
- Chao Dong, Chen Change Loy, Kaiming He and Xiaoou Tang. 2014. Learning a Deep Convolutional Network for Image Super-Resolution. In European Conference on Computer Vision. 184--199.Google ScholarCross Ref
- Jiwon Kim, Jung Kwon Lee and Kyoung Mu Lee. 2016. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1646--1654.Google ScholarCross Ref
- Tuzel Oncel, Taguchi Yuichi and Hershey John R. 2016. Global-Local Face Upsampling Network. arXiv:Computer Vision and Pattern Recognition.Google Scholar
- Qingxing Cao, Liang Lin, Yukai Shi, Xiaodan Liang and Guanbin Li. 2017. Attention-Aware Face Hallucination via Deep Reinforcement Learning. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1656--1664.Google ScholarCross Ref
- Huang G B, Mattar M and Berg T. 2008. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. Workshop on faces in'Real-Life'Images: detection, alignment, and recognition.Google Scholar
Index Terms
- A Lightweight Channel-spatial Attention Network for Real-time Image De-raining
Recommendations
Multifocal Attention-Based Cross-Scale Network for Image De-raining
MM '21: Proceedings of the 29th ACM International Conference on MultimediaAlbeit existing deep learning-based image de-raining methods have achieved promising results, most of them only extract single scale features, and neglect the fact that similar rain streaks appear repeatedly across different scales. Therefore, this ...
Non-locally Enhanced Encoder-Decoder Network for Single Image De-raining
MM '18: Proceedings of the 26th ACM international conference on MultimediaSingle image rain streaks removal has recently witnessed substantial progress due to the development of deep convolutional neural networks. However, existing deep learning based methods either focus on the entrance and exit of the network by decomposing ...
Image Inpainting Using Lightweight Transformer Neural Network Based on Channel Attention
ICNCC '23: Proceedings of the 2023 12th International Conference on Networks, Communication and ComputingImage inpainting is an important image processing application in repairing images with damaged or undesirable contents. While traditional approaches based on pixel or patch matching lack the ability to generate novel contents for the missing regions in ...
Comments