research-article

Robust Low-Rank Convolution Network for Image Denoising

Authors:

Meng WangAuthors Info & Claims

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

Pages 6211 - 6219

https://doi.org/10.1145/3503161.3547954

Published: 10 October 2022 Publication History

Abstract

Convolutional Neural Networks (CNNs) are powerful for image representation, but the convolution operation may be influenced and degraded by the included noise, and the deep features may not be fully learned. In this paper, we propose a new encoder-decoder based image restoration network, termed Robust Low-Rank Convolution Network with Feature Denoising (LRCnet). LRCnet presents a novel low-rank convolution (LR-Conv) for image representation, and a residual dense connection (RDC) for feature fusion between encoding and decoding. Different from directly splitting convolution into ordinary convolution and mirror convolution as existing work, LR-Conv deploys a feature denoising module after the ordinary convolution to remove noise for mirror convolution. A low-rank embedding process is then used to project the convolutional features into a robust low-rank subspace, which can retain the local geometry of input signal to some extent and separate the signal and noise by finding low-rank structure of features to reduce the impact of noise on convolution. Besides, most networks increase the depth of network simply to obtain deep information and lack of effective connections to fuse the multilevel features, which may not fully discover the deep features in various layers. Thus, we design a residual dense connection with a channel attention to connect multilevel feature effectively to obtain more useful information to enhance the data representation. Extensive experiments on several datasets verified the effectiveness of LRCnet for image denoising.

Supplementary Material

MP4 File (MM22-mmfp0842.mp4)

Presentation video

Download
17.98 MB

References

[1]

Abdelrahman Abdelhamed, Stephen Lin, and Michael S. Brown. 2018. A High-Quality Denoising Dataset for Smartphone Cameras. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018), 1692--1700.

[2]

Michal Aharon, Michael Elad, and Alfred Marcel Bruckstein. 2006. K-SVD: An Algorithm for Designing Overcomplete Dictionaries for Sparse Representation. IEEE Transactions on Signal Processing, Vol. 54, 1 (2006).

Digital Library

[3]

Saeed Anwar and Nick Barnes. 2019. Real Image Denoising With Feature Attention. In Proc. IEEE/CVF International Conference on Computer Vision (ICCV) (2019), 3155--3164.

[4]

Pablo Arbeláez, Michael Maire, Charlotte Fowlkes, and Julien Malik. 2011. Contour Detection and Hierarchical Image Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 33, 5 (2011), 898--916.

Digital Library

[5]

Antoni Buades, Bartomeu Coll, and Jean-Michel Morel. 2005. A non-local algorithm for image denoising. In Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR), Vol. 2 (2005), 60--65.

Digital Library

[6]

Harold Christopher Burger, Christian J. Schuler, and Stefan Harmeling. 2012. Image denoising: Can plain neural networks compete with BM3D? In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2012), 2392--2399.

[7]

Emmanuel J. Candès, Xiaodong Li, Yuliang Ma, and John Wright. 2011. Robust principal component analysis? J. ACM, Vol. 58 (2011), 11:1--11:37.

Digital Library

[8]

Jingwen Chen, Jiawei Chen, Hongyang Chao, and Ming Yang. 2018. Image Blind Denoising with Generative Adversarial Network Based Noise Modeling. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018), 3155--3164.

[9]

Kecheng Chen, Kun Long, Yazhou Ren, Jiayu Sun, and Xiaorong Pu. 2021. Lesion-Inspired Denoising Network: Connecting Medical Image Denoising and Lesion Detection. In Proceedings of the 29th ACM International Conference on Multimedia (ACM MM) (2021).

Digital Library

[10]

Yunjin Chen and Thomas Pock. 2017. Trainable Nonlinear Reaction Diffusion: A Flexible Framework for Fast and Effective Image Restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 39, 6 (2017).

Digital Library

[11]

Shen Cheng, Yuzhi Wang, Haibin Huang, Donghao Liu, Haoqiang Fan, and Shuaicheng Liu. 2021. NBNet: Noise Basis Learning for Image Denoising with Subspace Projection. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2021), 4894--4904.

[12]

Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen O. Egiazarian. 2007. Image Denoising by Sparse 3-D Transform-Domain Collaborative Filtering. IEEE Transactions on Image Processing, Vol. 16, 8 (2007).

[13]

Axel Davy, Thibaud Ehret, G. Facciolo, Jean-Michel Morel, and Pablo Arias. 2018. Non-Local Video Denoising by CNN. ArXiv, Vol. abs/1811.12758 (2018).

[14]

Weisheng Dong, Lei Zhang, Guangming Shi, and Xin Li. 2013. Nonlocally Centralized Sparse Representation for Image Restoration. IEEE Transactions on Image Processing, Vol. 22, 4 (2013).

Digital Library

[15]

Yuanbiao Gou, Peng Hu, Jiancheng Lv, and Xiaocui Peng. 2022. Multi-Scale Adaptive Network for Single Image Denoising. ArXiv, Vol. abs/2203.04313 (2022).

[16]

Shuhang Gu, Lei Zhang, Wangmeng Zuo, and Xiangchu Feng. 2014. Weighted Nuclear Norm Minimization with Application to Image Denoising. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2014), 2862--2869.

Digital Library

[17]

Shi Guo, Zifei Yan, K. Zhang, Wangmeng Zuo, and Lei Zhang. 2019. Toward Convolutional Blind Denoising of Real Photographs. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 1712--1722.

[18]

Kai Han, Yunhe Wang, Qi Tian, Jianyuan Guo, Chunjing Xu, and Chang Xu. 2020. GhostNet: More Features From Cheap Operations. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020), 1577--1586.

[19]

Jie Hu, Li Shen, Samuel Albanie, Gang Sun, and Enhua Wu. 2020. Squeeze-and-Excitation Networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 42, 8 (2020).

Digital Library

[20]

Viren Jain and H. Sebastian Seung. 2008. Natural Image Denoising with Convolutional Networks. In In Proc. Neural Information Processing Systems.

[21]

Jiwon Kim, Jung Kwon Lee, and Kyoung Mu Lee. 2016. Accurate Image Super-Resolution Using Very Deep Convolutional Networks. In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 1646--1654.

[22]

Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. In Proceedings of the International Conference on Learning Representations (ICLR), Vol. abs/1412.6980 (2015).

[23]

Stamatios Lefkimmiatis. 2018. Universal Denoising Networks : A Novel CNN Architecture for Image Denoising. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2018), 3204--3213.

[24]

Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, and Yuliang Ma. 2013. Robust Recovery of Subspace Structures by Low-Rank Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 35, 1 (2013).

Digital Library

[25]

Yu Liu and Zengfu Wang. 2015. Simultaneous image fusion and denoising with adaptive sparse representation. IET Image Process., Vol. 9 (2015), 347--357.

[26]

Kede Ma, Zhengfang Duanmu, Q. Wu, Zhou Wang, Hongwei Yong, Hongliang Li, and Lei Zhang. 2017. Waterloo Exploration Database: New Challenges for Image Quality Assessment Models. IEEE Transactions on Image Processing, Vol. 26, 2 (2017), 1004--1016.

Digital Library

[27]

Xiao-Jiao Mao, Chunhua Shen, and Yu-Bin Yang. 2016. Image Restoration Using Very Deep Convolutional Encoder-Decoder Networks with Symmetric Skip Connections. In In Proc. Neural Information Processing Systems.

[28]

Tobias Plötz and Stefan Roth. [n.d.]. Benchmarking Denoising Algorithms with Real Photographs. ( [n.,d.]).

[29]

Jiahuan Ren, Zhao Zhang, Jicong Fan, Haijun Zhang, Mingliang Xu, and Meng Wang. 2021. Robust Low-rank Deep Feature Recovery in CNNs: Toward Low Information Loss and Fast Convergence. 2021 IEEE International Conference on Data Mining (ICDM) (2021), 529--538.

[30]

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In In Proc. International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI).

[31]

Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Li Fei-Fei. 2015. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision, Vol. 115 (2015), 211--252.

Digital Library

[32]

Ying Tai, Jian Yang, Xiaoming Liu, and Chunyan Xu. 2017. MemNet: A Persistent Memory Network for Image Restoration. In Proc. IEEE International Conference on Computer Vision (ICCV) (2017), 4549--4557.

[33]

Graham M. Treece. 2022. Real Image Denoising With a Locally-Adaptive Bitonic Filter. IEEE Transactions on Image Processing, Vol. 31 (2022), 3151--3165.

[34]

Yanyan Wei, Zhao Zhang, Yang Wang, Mingliang Xu, Yi Yang, Shuicheng Yan, and Meng Wang. 2021. DerainCycleGAN: Rain Attentive CycleGAN for Single Image Deraining and Rainmaking. IEEE Transactions on Image Processing, Vol. 30 (2021), 4788--4801.

[35]

Cihang Xie, Yuxin Wu, Laurens van der Maaten, Alan Loddon Yuille, and Kaiming He. 2019. Feature Denoising for Improving Adversarial Robustness. In Proc. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2019), 501--509.

[36]

Jun Xu, Lei Zhang, and David Dian Zhang. 2018. A Trilateral Weighted Sparse Coding Scheme for Real-World Image Denoising. In Proc. European Conference on Computer Vision (ECCV), Vol. abs/1807.04364 (2018).

[37]

Jun Xu, Lei Zhang, David Dian Zhang, and Xiangchu Feng. 2017. Multi-channel Weighted Nuclear Norm Minimization for Real Color Image Denoising. In Proc. IEEE International Conference on Computer Vision (ICCV) (2017), 1105--1113.

[38]

Jun Xu, Lei Zhang, Wangmeng Zuo, David Zhang, and Xiangchu Feng. 2015. Patch Group Based Nonlocal Self-Similarity Prior Learning for Image Denoising. In Proc. IEEE International Conference on Computer Vision (ICCV) (2015), 244--252.

Digital Library

[39]

Zongsheng Yue, Hongwei Yong, Qian Zhao, L. M. Zhang, and Deyu Meng. 2019. Variational Denoising Network: Toward Blind Noise Modeling and Removal. In In Proc. Neural Information Processing Systems.

[40]

Zongsheng Yue, Qian Zhao, Lei Zhang, and Deyu Meng. 2020. Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generation. In Proc. European Conference on Computer Vision (ECCV), Vol. abs/2007.05946 (2020).

Digital Library

[41]

Syed Waqas Zamir, Aditya Arora, Salman Hameed Khan, Munawar Hayat, Fahad Shahbaz Khan, Ming-Hsuan Yang, and Ling Shao. 2020. Learning Enriched Features for Real Image Restoration and Enhancement. In Proc. European Conference on Computer Vision (ECCV) (2020).

Digital Library

[42]

Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. 2017. Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising. IEEE Transactions on Image Processing, Vol. 26, 7 (2017).

Digital Library

[43]

Kai Zhang, Wangmeng Zuo, and Lei Zhang. 2018. FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising. IEEE Transactions on Image Processing, Vol. 27, 9 (2018), 4608--4622.

[44]

Yulun Zhang, Yapeng Tian, Yu Kong, Bineng Zhong, and Yun Raymond Fu. 2021. Residual Dense Network for Image Restoration. IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 43, 7 (2021), 2480--2495.

[45]

Zhao Zhang, Fanzhang Li, Mingbo Zhao, Li Zhang, and Shuicheng Yan. 2016. Joint Low-Rank and Sparse Principal Feature Coding for Enhanced Robust Representation and Visual Classification. IEEE Transactions on Image Processing, Vol. 25 (2016), 2429--2443.

Digital Library

[46]

Zhao Zhang, Jiahuan Ren, Sheng Li, Richang Hong, Zhengjun Zha, and Meng Wang. 2019. Robust Subspace Discovery by Block-diagonal Adaptive Locality-constrained Representation. Proceedings of the 27th ACM International Conference on Multimedia (2019).

Digital Library

[47]

Huan Zheng, Zhao Zhang, Yang Wang, Zheng Zhang, Mingliang Xu, Yi Yang, and Meng Wang. 2021. GCM-Net: Towards Effective Global Context Modeling for Image Inpainting. Proceedings of the 29th ACM International Conference on Multimedia (2021).

Digital Library

[48]

Yifeng Zhou, Xing Xu, Fumin Shen, Lianli Gao, Huimin Lu, and Heng Tao Shen. 2020. Temporal Denoising Mask Synthesis Network for Learning Blind Video Temporal Consistency. In Proceedings of the 28th ACM International Conference on Multimedia (ACM MM) (2020).

Digital Library

[49]

Yaochen Zhu, Zhenzhong Chen, and Feng Wu. 2019. Multimodal Deep Denoise Framework for Affective Video Content Analysis. In Proceedings of the 27th ACM International Conference on Multimedia (ACM MM) (2019).

Digital Library

Cited By

Ren JZhang ZZhao SFan JZhao ZZhao YHong RWang M(2025)When low-light meets flares: Towards Synchronous Flare Removal and Brightness EnhancementNeural Networks10.1016/j.neunet.2025.107149185(107149)Online publication date: May-2025
https://doi.org/10.1016/j.neunet.2025.107149
Jiang BLi JLu YCai QSong HLu G(2025)Eficient image denoising using deep learning: A brief surveyInformation Fusion10.1016/j.inffus.2025.103013118(103013)Online publication date: Jun-2025
https://doi.org/10.1016/j.inffus.2025.103013
Ding XGong YShi THuang ZXu GYang XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Masked Snake Attention for Fundus Image Restoration with Vessel PreservationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680722(4368-4376)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680722
Show More Cited By

Index Terms

Robust Low-Rank Convolution Network for Image Denoising
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision representations
        Hierarchical representations

Recommendations

A Multi-scale Dilated Residual Convolution Network for Image Denoising
Abstract
Due to the excellent performance of deep learning, more and more image denoising methods based on convolutional neural networks (CNN) are proposed, including dilated convolution method and multi-scale convolution method. A fundamental issue is how ...
Nonlocal low-rank plus deep denoising prior for robust image compressed sensing reconstruction
Abstract
It is challenging for current compressive sensing (CS) approaches to reconstruct image from compressed observations with impulsive noise and outliers, termed robust image CS problem. In this paper, we propose a novel reconstruction model for the ...
Highlights
- A robust image CS reconstruction approach with nonlocal low-rank plus deep denoising prior is proposed.
- An efficient optimization framework based on half-quadratic (HQ) theory and the alternative minimizing scheme is developed.
- The ...
Image denoising with complex ridgelets

In this paper, we propose a novel image denoising method by incorporating the dual-tree complex wavelets into the ordinary ridgelet transform. The approximate shift invariant property of the dual-tree complex wavelet and the high directional sensitivity ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

MM '22: Proceedings of the 30th ACM International Conference on Multimedia

October 2022

7537 pages

ISBN:9781450392037

DOI:10.1145/3503161

General Chairs:
João Magalhães
NOVA University of Lisbon, Portugal
,
Alberto del Bimbo
University of Florence, Italy
,
Shin'ichi Satoh
National Institute of Informatics, Japan
,
Nicu Sebe
University of Trento, Italy
,
Program Chairs:
Xavier Alameda-Pineda
Inria, Grenoble, France
,
Qin Jin
Renmin University of China, China
,
Vincent Oria
New Jersey Institute of Technology, USA
,
Laura Toni
University College London, UK

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 October 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Conference

MM '22

Sponsor:

SIGMM

MM '22: The 30th ACM International Conference on Multimedia

October 10 - 14, 2022

Lisboa, Portugal

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

12
Total Citations
View Citations
399
Total Downloads

Downloads (Last 12 months)85
Downloads (Last 6 weeks)4

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Ren JZhang ZZhao SFan JZhao ZZhao YHong RWang M(2025)When low-light meets flares: Towards Synchronous Flare Removal and Brightness EnhancementNeural Networks10.1016/j.neunet.2025.107149185(107149)Online publication date: May-2025
https://doi.org/10.1016/j.neunet.2025.107149
Jiang BLi JLu YCai QSong HLu G(2025)Eficient image denoising using deep learning: A brief surveyInformation Fusion10.1016/j.inffus.2025.103013118(103013)Online publication date: Jun-2025
https://doi.org/10.1016/j.inffus.2025.103013
Ding XGong YShi THuang ZXu GYang XCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Masked Snake Attention for Fundus Image Restoration with Vessel PreservationProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680722(4368-4376)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680722
Makwana DDeshmukh GSusladkar OMittal SR S(2024)LIVENet: A novel network for real-world low-light image denoising and enhancement2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00575(5844-5853)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00575
Ji ZZheng HZhang ZYe QZhao YXu M(2024)Multi-Scale Interaction Network for Low-Light Stereo Image EnhancementIEEE Transactions on Consumer Electronics10.1109/TCE.2023.328022970:1(3626-3634)Online publication date: Feb-2024
https://doi.org/10.1109/TCE.2023.3280229
Liang GLai ZKong H(2024)Support vector machine with discriminative low‐rank embeddingCAAI Transactions on Intelligence Technology10.1049/cit2.123299:5(1249-1262)Online publication date: 2-Apr-2024
https://dl.acm.org/doi/10.1049/cit2.12329
Xu LKawakami RInoue NEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)Scale-space Tokenization for Improving the Robustness of Vision TransformersProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612060(2684-2693)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612060
Li YZhang ZZhao SFan JZhang HXu M(2023)Feature Aggregating Network with Inter-Frame Interaction for Efficient Video Super-Resolution2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00042(329-338)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00042
Lu HYu ZNiu XChen Y(2023)Neuron Structure Modeling for Generalizable Remote Physiological Measurement2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01783(18589-18599)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01783
Tian ZChen JZhang X(2023)On-the-fly Improving Performance of Deep Code Models via Input Denoising2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)10.1109/ASE56229.2023.00166(560-572)Online publication date: 11-Sep-2023
https://doi.org/10.1109/ASE56229.2023.00166
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten