Abstract
Convolutional neural network (CNN) has shown its superpower in image denoising in recent years. However, most CNN models suffer from a large number of model parameters and the effect of image denoising still needs to be improved. To cope with these issues, we propose a recursive lightweight CNN approach that can make the noisy images purer and purer, namely PPNets, in this paper. The PPNets mainly consist of four parts: separable convolution–batch normalization–ReLU (SCBR) blocks to extract coarse features, bottlenecks with skip connection to integrate coarse features and refined features to enhance expression ability of model, noise proposal network with an attention mechanism to predict noise level and recursive strategies to stack the denoising model to make the noisy images purer and purer. Since SCBR uses depthwise convolution and pointwise convolution to replace traditional convolution operations, the proposed PPNets have fewer weight parameters. We conduct extensive experiments on two gray image datasets and three color image datasets. The experimental results demonstrate that the PPNets are significantly superior to the traditional models in denoising effectiveness. At the same time, the PPNets outperform the compared state-of-the-art CNN models in terms of both denoising effectiveness and the number of model parameters.












Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
All data generated or analyzed during this study are included in this published article.
References
Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian Denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)
Goyal, B., Dogra, A., Agrawal, S., Sohi, B.S., Sharma, A.: Image denoising review: from classical to state-of-the-art approaches. Inf. Fusion 55, 220–244 (2020)
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 839–846. IEEE (1998)
Benesty, J., Chen, J., Huang, Y.: Study of the widely linear wiener filter for noise reduction. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 205–208. IEEE (2010)
Zhang, M., Gunturk, B.K.: Multiresolution bilateral filtering for image denoising. IEEE Trans. Image Process. 17(12), 2324–2333 (2008)
Buades, A., Coll, B., Morel, J.-M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)
Wang, S., Xia, Y., Liu, Q., Luo, J., Zhu, Y., Feng, D.D.: Gabor feature based nonlocal means filter for textured image denoising. J. Vis. Commun. Image Represent. 23(7), 1008–1018 (2012)
Zhang, X., Feng, X., Wang, W.: Two-direction nonlocal model for image denoising. IEEE Trans. Image Process. 22(1), 408–412 (2012)
Tian, C., Zheng, M., Zuo, W., Zhang, B., Zhang, Y., Zhang, D.: Multi-stage image denoising with the wavelet transform. Pattern Recognit. 134, 109050 (2022)
Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)
Zhuang Zhang, X., Chen, L.L., Li, Y., Deng, Y.: A sparse representation denoising algorithm for visible and infrared image based on orthogonal matching pursuit. SIViP 14(4), 737–745 (2020)
Mahdaoui, A.E., Ouahabi, A., Moulay, M.S.: Image denoising using a compressive sensing approach based on regularization constraints. Sensors 22(6), 2199 (2022)
Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)
Zhang, M., Desrosiers, C.: Image denoising based on sparse representation and gradient histogram. IET Image Process 11(1), 54–63 (2017)
Liu, H., Li, L., Jiangbo, L., Tan, S.: Group sparsity mixture model and its application on image denoising. IEEE Trans. Image Process. 31, 5677–5690 (2022)
Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imag. Vis. 20(1), 89–97 (2004)
Fang, F., Li, F., Zeng, T.: Single image dehazing and denoising: a fast variational approach. SIAM J. Imag. Sci. 7(2), 969–996 (2014)
Islam, M.R., Xu, C., Han, Y., Ashfaq, R.A.R.: A novel weighted variational model for image denoising. Int. J. Pattern Recognit. Artif. Intel. 31(12), 17540222 (2017)
Tian, C., Yong, X., Li, Z., Zuo, W., Fei, L., Liu, H.: Attention-guided CNN for image denoising. Neural Netw. 124, 117–129 (2020)
Zhang, K., Zuo, W., Zhang, L.: FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)
Zhang, Y., Li, K., Li, K., Sun, G., Kong, Y., Fu, Y.: Accurate and fast image denoising via attention guided scaling. IEEE Trans. Image Process. 30, 6255–6265 (2021)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 234–241. Springer (2015)
Qiao, S., Yang, J., Zhang, T., Zhao, C.: Layered input GradiNet for image denoising. Knowl.-Based Syst. 254, 109587 (2022)
Zhang, J., Cao, L., Wang, T., Wenlong, F., Shen, W.: NHNet: a non-local hierarchical network for image denoising. IET Image Proc. 16(9), 2446–2456 (2022)
Jia, F., Wong, W.H., Zeng, T.: DDUNet: Dense dense U-net with applications in image denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 354–364. IEEE (2021)
Bai, Yu., Liu, M., Yao, C., Lin, C., Zhao, Y.: Multi-stage progressive network for image denoising. Neurocomputing, MSPNet (2022)
Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 3147–3155. IEEE (2017)
Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2472–2481. IEEE (2018)
Jia, X., Chai, H., Guo, Y., Huang, Y., Zhao, B.: Multiscale parallel feature extraction convolution neural network for image denoising. J. Electron. Imaging 27(6), 063031 (2018)
Quan, Y., Chen, Y., Shao, Y., Teng, H., Yong, X., Ji, H.: Image denoising using complex-valued deep CNN. Pattern Recogn. 111, 107639 (2021)
Rawat, S., Rana, K.P.S., Kumar, V.: A novel complex-valued convolutional neural network for medical image denoising. Biomed. Signal Process. Control 69, 102859 (2021)
Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: Can plain neural networks compete with BM3D? In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2392–2399. IEEE (2012)
Jain, V., Seung, S.: Natural image denoising with convolutional networks. In Proceedings of the 21st International Conference on Neural Information Processing Systems (NIPS), NIPS’08, pp. 769–776, Red Hook, NY, USA, Curran Associates Inc (2008)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 25, 1097–1105 (2012)
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 1–9. IEEE (2015)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 770–778. IEEE (2016)
Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 4549–4557. IEEE (2017)
Tian, C., Yong, X., Zuo, W.: Image denoising using deep CNN with batch renormalization. Neural Netw. 121, 461–473 (2020)
Chen, C., Xiong, Z., Tian, X., Wu, F.: Deep boosting for image denoising. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–18. Springer (2018)
Zuo, Z., Chen, X., Xu, H., Li, J., Liao, W., Yang, Z.X., Wang, S.: IDEA-Net: adaptive dual self-attention network for single image denoising. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 739–748. IEEE (2022)
Jia, F., Ma, L., Yang, Y., Zeng, T.: Pixel-attention CNN with color correlation loss for color image denoising. IEEE Signal Process. Lett. 28, 1600–1604 (2021)
Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 3929–3938. IEEE (2017)
Huang, J., Zhao, Z., Ren, C., Teng, Q., He, X.: A prior-guided deep network for real image denoising and its applications. Knowl.-Based Syst. 255, 109776 (2022)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 1251–1258. IEEE (2017)
Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)
Li, G., Jiang, S., Yun, I., Kim, J., Kim, J.: Depth-wise asymmetric bottleneck with point-wise aggregation decoder for real-time semantic segmentation in urban scenes. IEEE Access 8, 27495–27506 (2020)
Wang, J., Xiong, H., Wang, H., Nian, X.: ADSCNet: asymmetric depthwise separable convolution for semantic segmentation in real-time. Appl. Intell. 50(4), 1045–1056 (2020)
Xuan, L., Un, K.-F., Lam, C.-S., Martins, R.P.: An FPGA-based energy-efficient reconfigurable depthwise separable convolution accelerator for image recognition. IEEE Trans. Circuits Syst. II Express Briefs 69(10), 4003–4007 (2022)
Li, W., Chen, H., Liu, Q., Liu, H., Wang, Y., Gui, G.: Attention mechanism and depthwise separable convolution aided 3DCNN for hyperspectral remote sensing image classification. Remote Sens. 14(9), 2215 (2022)
Li, X., Zhengshun, D., Huang, Y., Tan, Z.: A deep translation (GAN) based change detection network for optical and SAR remote sensing images. ISPRS J. Photogramm. Remote. Sens. 179, 14–34 (2021)
Muhammad, W., Aramvith, S., Onoye, T.: Multi-scale xception based depthwise separable convolution for single image super-resolution. PLoS ONE 16(8), e0249278 (2021)
Liu, G., Dang, M., Liu, J., Xiang, R., Tian, Y., Luo, N.: True wide convolutional neural network for image denoising. Inf. Sci. 610, 171–184 (2022)
Mao, X., Shen, C., Yang, Y.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Adv. Neural. Inf. Process. Syst. 29, 2802–2810 (2016)
Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 860–867. IEEE (2005)
Franzen, R.: Kodak lossless true color image suite. http://r0k.us/graphics/kodak (1999)
Zhang, L., Xiaolin, W., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. Imaging 20(20), 023016 (2011)
Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 416–423. IEEE (2001)
Ma, K., Duanmu, Z., Qingbo, W., Wang, Z., Yong, H., Li, H., Zhang, L.: Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans. Image Process. 26(2), 1004–1016 (2016)
He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1026–1034. IEEE (2015)
Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2862–2869. IEEE (2014)
Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 479–486. IEEE (2011)
Schmidt, U., Roth, S.: Shrinkage fields for effective image restoration. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2774–2781. IEEE (2014)
Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256–1272 (2016)
Tian, C., Yong, X., Fei, L., Wang, J., Wen, J., Luo, N.: Enhanced CNN for image denoising. CAAI Trans. Intell. Technol. 4(1), 17–23 (2019)
Acknowledgements
This work was supported by the Ministry of Education of Humanities and Social Science Project (Grant no. 19YJAZH047) and the Scientific Research Fund of Sichuan Provincial Education Department (Grant no. 17ZB0433).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Shi, J., Li, T. & Xu, J. Recursive lightweight convolutional neural networks that make noisy images purer and purer. Vis Comput 39, 6571–6587 (2023). https://doi.org/10.1007/s00371-022-02749-y
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-022-02749-y