Skip to main content
SpringerLink
Log in

Recursive lightweight convolutional neural networks that make noisy images purer and purer

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

Convolutional neural network (CNN) has shown its superpower in image denoising in recent years. However, most CNN models suffer from a large number of model parameters and the effect of image denoising still needs to be improved. To cope with these issues, we propose a recursive lightweight CNN approach that can make the noisy images purer and purer, namely PPNets, in this paper. The PPNets mainly consist of four parts: separable convolution–batch normalization–ReLU (SCBR) blocks to extract coarse features, bottlenecks with skip connection to integrate coarse features and refined features to enhance expression ability of model, noise proposal network with an attention mechanism to predict noise level and recursive strategies to stack the denoising model to make the noisy images purer and purer. Since SCBR uses depthwise convolution and pointwise convolution to replace traditional convolution operations, the proposed PPNets have fewer weight parameters. We conduct extensive experiments on two gray image datasets and three color image datasets. The experimental results demonstrate that the PPNets are significantly superior to the traditional models in denoising effectiveness. At the same time, the PPNets outperform the compared state-of-the-art CNN models in terms of both denoising effectiveness and the number of model parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data Availability

All data generated or analyzed during this study are included in this published article.

References

  1. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a Gaussian Denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    MathSciNet  MATH  Google Scholar 

  2. Goyal, B., Dogra, A., Agrawal, S., Sohi, B.S., Sharma, A.: Image denoising review: from classical to state-of-the-art approaches. Inf. Fusion 55, 220–244 (2020)

    Google Scholar 

  3. Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 839–846. IEEE (1998)

  4. Benesty, J., Chen, J., Huang, Y.: Study of the widely linear wiener filter for noise reduction. In IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 205–208. IEEE (2010)

  5. Zhang, M., Gunturk, B.K.: Multiresolution bilateral filtering for image denoising. IEEE Trans. Image Process. 17(12), 2324–2333 (2008)

    MathSciNet  MATH  Google Scholar 

  6. Buades, A., Coll, B., Morel, J.-M.: A review of image denoising algorithms, with a new one. Multiscale Model. Simul. 4(2), 490–530 (2005)

    MathSciNet  MATH  Google Scholar 

  7. Wang, S., Xia, Y., Liu, Q., Luo, J., Zhu, Y., Feng, D.D.: Gabor feature based nonlocal means filter for textured image denoising. J. Vis. Commun. Image Represent. 23(7), 1008–1018 (2012)

    Google Scholar 

  8. Zhang, X., Feng, X., Wang, W.: Two-direction nonlocal model for image denoising. IEEE Trans. Image Process. 22(1), 408–412 (2012)

    MathSciNet  MATH  Google Scholar 

  9. Tian, C., Zheng, M., Zuo, W., Zhang, B., Zhang, Y., Zhang, D.: Multi-stage image denoising with the wavelet transform. Pattern Recognit. 134, 109050 (2022)

    Google Scholar 

  10. Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)

    MathSciNet  Google Scholar 

  11. Zhuang Zhang, X., Chen, L.L., Li, Y., Deng, Y.: A sparse representation denoising algorithm for visible and infrared image based on orthogonal matching pursuit. SIViP 14(4), 737–745 (2020)

    Google Scholar 

  12. Mahdaoui, A.E., Ouahabi, A., Moulay, M.S.: Image denoising using a compressive sensing approach based on regularization constraints. Sensors 22(6), 2199 (2022)

    Google Scholar 

  13. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    MathSciNet  Google Scholar 

  14. Zhang, M., Desrosiers, C.: Image denoising based on sparse representation and gradient histogram. IET Image Process 11(1), 54–63 (2017)

    Google Scholar 

  15. Liu, H., Li, L., Jiangbo, L., Tan, S.: Group sparsity mixture model and its application on image denoising. IEEE Trans. Image Process. 31, 5677–5690 (2022)

    Google Scholar 

  16. Chambolle, A.: An algorithm for total variation minimization and applications. J. Math. Imag. Vis. 20(1), 89–97 (2004)

    MathSciNet  MATH  Google Scholar 

  17. Fang, F., Li, F., Zeng, T.: Single image dehazing and denoising: a fast variational approach. SIAM J. Imag. Sci. 7(2), 969–996 (2014)

    MathSciNet  MATH  Google Scholar 

  18. Islam, M.R., Xu, C., Han, Y., Ashfaq, R.A.R.: A novel weighted variational model for image denoising. Int. J. Pattern Recognit. Artif. Intel. 31(12), 17540222 (2017)

    Google Scholar 

  19. Tian, C., Yong, X., Li, Z., Zuo, W., Fei, L., Liu, H.: Attention-guided CNN for image denoising. Neural Netw. 124, 117–129 (2020)

    Google Scholar 

  20. Zhang, K., Zuo, W., Zhang, L.: FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)

    MathSciNet  Google Scholar 

  21. Zhang, Y., Li, K., Li, K., Sun, G., Kong, Y., Fu, Y.: Accurate and fast image denoising via attention guided scaling. IEEE Trans. Image Process. 30, 6255–6265 (2021)

    Google Scholar 

  22. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI), pp. 234–241. Springer (2015)

  23. Qiao, S., Yang, J., Zhang, T., Zhao, C.: Layered input GradiNet for image denoising. Knowl.-Based Syst. 254, 109587 (2022)

    Google Scholar 

  24. Zhang, J., Cao, L., Wang, T., Wenlong, F., Shen, W.: NHNet: a non-local hierarchical network for image denoising. IET Image Proc. 16(9), 2446–2456 (2022)

    Google Scholar 

  25. Jia, F., Wong, W.H., Zeng, T.: DDUNet: Dense dense U-net with applications in image denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 354–364. IEEE (2021)

  26. Bai, Yu., Liu, M., Yao, C., Lin, C., Zhao, Y.: Multi-stage progressive network for image denoising. Neurocomputing, MSPNet (2022)

    Google Scholar 

  27. Tai, Y., Yang, J., Liu, X.: Image super-resolution via deep recursive residual network. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 3147–3155. IEEE (2017)

  28. Zhang, Y., Tian, Y., Kong, Y., Zhong, B., Fu, Y.: Residual dense network for image super-resolution. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2472–2481. IEEE (2018)

  29. Jia, X., Chai, H., Guo, Y., Huang, Y., Zhao, B.: Multiscale parallel feature extraction convolution neural network for image denoising. J. Electron. Imaging 27(6), 063031 (2018)

    Google Scholar 

  30. Quan, Y., Chen, Y., Shao, Y., Teng, H., Yong, X., Ji, H.: Image denoising using complex-valued deep CNN. Pattern Recogn. 111, 107639 (2021)

    Google Scholar 

  31. Rawat, S., Rana, K.P.S., Kumar, V.: A novel complex-valued convolutional neural network for medical image denoising. Biomed. Signal Process. Control 69, 102859 (2021)

    Google Scholar 

  32. Burger, H.C., Schuler, C.J., Harmeling, S.: Image denoising: Can plain neural networks compete with BM3D? In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2392–2399. IEEE (2012)

  33. Jain, V., Seung, S.: Natural image denoising with convolutional networks. In Proceedings of the 21st International Conference on Neural Information Processing Systems (NIPS), NIPS’08, pp. 769–776, Red Hook, NY, USA, Curran Associates Inc (2008)

  34. Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Commun. ACM 25, 1097–1105 (2012)

    Google Scholar 

  35. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 1–9. IEEE (2015)

  36. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 770–778. IEEE (2016)

  37. Tai, Y., Yang, J., Liu, X., Xu, C.: Memnet: a persistent memory network for image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 4549–4557. IEEE (2017)

  38. Tian, C., Yong, X., Zuo, W.: Image denoising using deep CNN with batch renormalization. Neural Netw. 121, 461–473 (2020)

    Google Scholar 

  39. Chen, C., Xiong, Z., Tian, X., Wu, F.: Deep boosting for image denoising. In Proceedings of the European Conference on Computer Vision (ECCV), pp. 3–18. Springer (2018)

  40. Zuo, Z., Chen, X., Xu, H., Li, J., Liao, W., Yang, Z.X., Wang, S.: IDEA-Net: adaptive dual self-attention network for single image denoising. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 739–748. IEEE (2022)

  41. Jia, F., Ma, L., Yang, Y., Zeng, T.: Pixel-attention CNN with color correlation loss for color image denoising. IEEE Signal Process. Lett. 28, 1600–1604 (2021)

    Google Scholar 

  42. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 3929–3938. IEEE (2017)

  43. Huang, J., Zhao, Z., Ren, C., Teng, Q., He, X.: A prior-guided deep network for real image denoising and its applications. Knowl.-Based Syst. 255, 109776 (2022)

    Google Scholar 

  44. Chollet, F.: Xception: deep learning with depthwise separable convolutions. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 1251–1258. IEEE (2017)

  45. Howard, A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Weyand, T., Andreetto, M., Adam, H.: Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  46. Li, G., Jiang, S., Yun, I., Kim, J., Kim, J.: Depth-wise asymmetric bottleneck with point-wise aggregation decoder for real-time semantic segmentation in urban scenes. IEEE Access 8, 27495–27506 (2020)

    Google Scholar 

  47. Wang, J., Xiong, H., Wang, H., Nian, X.: ADSCNet: asymmetric depthwise separable convolution for semantic segmentation in real-time. Appl. Intell. 50(4), 1045–1056 (2020)

    Google Scholar 

  48. Xuan, L., Un, K.-F., Lam, C.-S., Martins, R.P.: An FPGA-based energy-efficient reconfigurable depthwise separable convolution accelerator for image recognition. IEEE Trans. Circuits Syst. II Express Briefs 69(10), 4003–4007 (2022)

    Google Scholar 

  49. Li, W., Chen, H., Liu, Q., Liu, H., Wang, Y., Gui, G.: Attention mechanism and depthwise separable convolution aided 3DCNN for hyperspectral remote sensing image classification. Remote Sens. 14(9), 2215 (2022)

    Google Scholar 

  50. Li, X., Zhengshun, D., Huang, Y., Tan, Z.: A deep translation (GAN) based change detection network for optical and SAR remote sensing images. ISPRS J. Photogramm. Remote. Sens. 179, 14–34 (2021)

    Google Scholar 

  51. Muhammad, W., Aramvith, S., Onoye, T.: Multi-scale xception based depthwise separable convolution for single image super-resolution. PLoS ONE 16(8), e0249278 (2021)

    Google Scholar 

  52. Liu, G., Dang, M., Liu, J., Xiang, R., Tian, Y., Luo, N.: True wide convolutional neural network for image denoising. Inf. Sci. 610, 171–184 (2022)

    Google Scholar 

  53. Mao, X., Shen, C., Yang, Y.: Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. Adv. Neural. Inf. Process. Syst. 29, 2802–2810 (2016)

    Google Scholar 

  54. Roth, S., Black, M.J.: Fields of experts: a framework for learning image priors. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 860–867. IEEE (2005)

  55. Franzen, R.: Kodak lossless true color image suite. http://r0k.us/graphics/kodak (1999)

  56. Zhang, L., Xiaolin, W., Buades, A., Li, X.: Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J. Electron. Imaging 20(20), 023016 (2011)

    Google Scholar 

  57. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 416–423. IEEE (2001)

  58. Ma, K., Duanmu, Z., Qingbo, W., Wang, Z., Yong, H., Li, H., Zhang, L.: Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans. Image Process. 26(2), 1004–1016 (2016)

    MathSciNet  MATH  Google Scholar 

  59. He, K., Zhang, X., Ren, S., Sun, J.: Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 1026–1034. IEEE (2015)

  60. Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2862–2869. IEEE (2014)

  61. Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 479–486. IEEE (2011)

  62. Schmidt, U., Roth, S.: Shrinkage fields for effective image restoration. In Proceedings of the IEEE/CVF International Conference on Computer Vision (CVPR), pp. 2774–2781. IEEE (2014)

  63. Chen, Y., Pock, T.: Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1256–1272 (2016)

    Google Scholar 

  64. Tian, C., Yong, X., Fei, L., Wang, J., Wen, J., Luo, N.: Enhanced CNN for image denoising. CAAI Trans. Intell. Technol. 4(1), 17–23 (2019)

    Google Scholar 

Download references

Acknowledgements

This work was supported by the Ministry of Education of Humanities and Social Science Project (Grant no. 19YJAZH047) and the Scientific Research Fund of Sichuan Provincial Education Department (Grant no. 17ZB0433).

Author information

Authors and Affiliations

  1. School of Computing and Artificial Intelligence, Southwestern University of Finance and Economics, Chengdu, 611130, China

    Jiayi Shi, Taiyong Li & Jiaxuan Xu

Authors
  1. Jiayi Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Taiyong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiaxuan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Taiyong Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shi, J., Li, T. & Xu, J. Recursive lightweight convolutional neural networks that make noisy images purer and purer. Vis Comput 39, 6571–6587 (2023). https://doi.org/10.1007/s00371-022-02749-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-022-02749-y

Keywords

Search

Navigation