Skip to main content
SpringerLink
Log in

A new multi-scale CNN with pixel-wise attention for image denoising

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Convolutional neural networks (CNNs) are widely popular in the field of image denoising. A large number of CNN-based denoising methods exhibit superior denoising performance in comparison with most conventional denoising schemes. However, some of these approaches extract the noise by stacking many common convolutional layers, which makes them prone to overfitting and causes more loss of image details since the erroneous extraction of non-noise features. A new multi-scale denoising network (MSDNet) is proposed for better tackling these issues, which uses the multi-scale feature information and pixel-wise correlation to effectively remove more noise from noisy images and retain more image details. The denoising effectiveness of MSDNet is specifically attributed to its three key modules, namely multi-scale progressive fusion block (MSPFB), pixel-wise attention block (PWAB) and residual learning (RL), in which MSPFB helps MSDNet capture more useful context information and reduce important information loss caused by ignoring scale inconsistency for capturing more noise from noisy images while maintaining more image details, PWAB facilitates MSDNet to selectively focus on specific image pixels or regions for further effectively capturing noise from noisy images while better preserving image details, and RL is helpful for MSDNet to better address deeper neural network training difficulties and mitigate overfitting. Experimental results demonstrate that MSDNet exhibits superior denoising and single-image deraining performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  2. Zoran, D., Weiss, Y.: From learning models of natural image patches to whole image restoration. In: International Conference on Computer Vision, pp. 479–486 (2011)

  3. Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2862–2869 (2014)

  4. Herbreteau, S., Kervrann, C.: Towards a unified view of unsupervised non-local methods for image denoising: The NL-Ridge approach. In: IEEE International Conference on Image Processing, pp. 3376–3380 (2022)

  5. Deng, H., Liu, G., Zhou, L.: Ultrasonic logging image denoising algorithm based on variational Bayesian and sparse prior. J. Electron. Imag. 32(1), 013004–013004 (2023)

    Article  ADS  Google Scholar 

  6. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  7. Zhang, K., Zuo, W., Gu, S., Zhang, L.: Learning deep CNN denoiser prior for image restoration. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3929–3938 (2017)

  8. Tian, C., Xu, Y., Fei, L., Wang, J., Wen, J., Luo, N.: Enhanced CNN for image denoising. CAAI Trans. Intell. Technol. 4(1), 17–23 (2019)

    Article  Google Scholar 

  9. Peng, Y., Zhang, L., Liu, S., Wu, X., Zhang, Y., Wang, X.: Dilated residual networks with symmetric skip connection for image denoising. Neurocomputing 345, 67–76 (2019)

    Article  Google Scholar 

  10. Zhang, L., Li, Y., Wang, P., Wei, W., Xu, S., Zhang, Y.: A separation-aggregation network for image denoising. Appl. Soft Comput. 83, 105603 (2019)

    Article  Google Scholar 

  11. Zhang, Q., Xiao, J., Tian, C., Chun-Wei Lin, J., Zhang, S.: A robust deformed convolutional neural network (CNN) for image denoising. CAAI Trans. Intell. Technol. 8(2), 331–342 (2023)

    Article  Google Scholar 

  12. Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: International Conference on Machine Learning, pp. 448–456 (2015)

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  14. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. In: International Conference on Learning Representations, pp. 1–13 (2016)

  15. Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., Wei, Y.: Deformable convolutional networks. In: IEEE International Conference on Computer Vision, pp. 764–773 (2017)

  16. Liang, J., Cao, J., Sun, G., Zhang, K., Van Gool, L., Timofte, R.: SwinIR: Image restoration using swin Transformer. In: IEEE/CVF International Conference on Computer Vision, pp. 1833–1844 (2021)

  17. Fan, C.-M., Liu, T.-J., Liu, K.-H.: SUNet: Swin Transformer UNet for image denoising. In: IEEE International Symposium on Circuits Systems, pp. 2333–2337 (2022)

  18. Wan, Y., Shao, M., Cheng, Y., Meng, D., Zuo, W.: Progressive convolutional transformer for image restoration. Eng. Appl. Artif. Intell. 125, 106755 (2023)

    Article  Google Scholar 

  19. Liu, P., Zhang, H., Zhang, K., Lin, L., Zuo, W.: Multi-level wavelet-CNN for image restoration. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 773–782 (2018)

  20. Peng, Y., Cao, Y., Liu, S., Yang, J., Zuo, W.: Progressive training of multi-level wavelet residual networks for image denoising. arXiv preprint arXiv:2010.12422 (2020)

  21. Tian, C., Zheng, M., Zuo, W., Zhang, B., Zhang, Y., Zhang, D.: Multi-stage image denoising with the wavelet transform. Pattern Recognit. 134, 109050 (2023)

    Article  Google Scholar 

  22. Deng, S., Wei, M., Wang, J., Feng, Y., Liang, L., Xie, H., Wang, F.L., Wang, M.: Detail-recovery image deraining via context aggregation networks. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14548–14557 (2020)

  23. Zhang, H., Xie, Q., Lu, B., Gai, S.: Dual attention residual group networks for single image deraining. Dig. Signal Process. 116, 103106 (2021)

    Article  Google Scholar 

  24. Bin, H., Jinhang, L., Lili, Z., Shi, C.: Split frequency attention network for single image deraining. Signal Image Video Process. 17, 3741–3748 (2023)

    Article  Google Scholar 

  25. Mi, Z., Jiang, X., Sun, T., Xu, K.: GAN-generated image detection with self-attention mechanism against GAN generator defect. IEEE J. Select. Top. Signal Process. 14(5), 969–981 (2020)

    Article  ADS  Google Scholar 

  26. Guo, Y., Zhou, L.: Mea-net: a lightweight SAR ship detection model for imbalanced datasets. Remot. Sens. 14(18), 4438 (2022)

    Article  ADS  Google Scholar 

  27. Zhang, M., Liu, Z., Feng, J., Liu, L., Jiao, L.: Remote sensing image change detection based on deep multi-scale multi-attention Siamese Transformer network. Remot. Sens. 15(3), 842 (2023)

    Article  ADS  Google Scholar 

  28. Tian, C., Xu, Y., Li, Z., Zuo, W., Fei, L., Liu, H.: Attention-guided CNN for image denoising. Neural Netw. 124, 117–129 (2020)

    Article  PubMed  Google Scholar 

  29. Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: 3rd International Conference on Learning Representations (2015)

  30. Ephraim, Y., Malah, D.: Speech enhancement using a minimum-mean square error short-time spectral amplitude estimator. IEEE Trans. Acoust. Speech Signal Process. 32(6), 1109–1121 (1984)

    Article  Google Scholar 

  31. Nam, S., Hwang, Y., Matsushita, Y., Kim, S.J.: A holistic approach to cross-channel image noise modeling and its application to image denoising. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1683–1691 (2016)

  32. Tian, C., Xu, Y., Zuo, W., Du, B., Lin, C.-W., Zhang, D.: Designing and training of a dual CNN for image denoising. Knowl. Syst. 226, 106949 (2021)

    Article  Google Scholar 

  33. Luo, Y., Xu, Y., Ji, H.: Removing rain from a single image via discriminative sparse coding. In: IEEE International Conference on Computer Vision, pp. 3397–3405 (2015)

  34. Deng, L.-J., Huang, T.-Z., Zhao, X.-L., Jiang, T.-X.: A directional global sparse model for single image rain removal. Appl. Math. Modell. 59, 662–679 (2018)

  35. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., Paisley, J.: Removing rain from single images via a deep detail network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3855–3863 (2017)

  36. Fu, X., Huang, J., Ding, X., Liao, Y., Paisley, J.: Clearing the skies: a deep network architecture for single-image rain removal. IEEE Trans. Image Process. 26(6), 2944–2956 (2017)

    Article  ADS  MathSciNet  Google Scholar 

  37. Zhang, H., Patel, V.M.: Density-aware single image de-raining using a multi-stream dense network. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 695–704 (2018)

  38. Jiang, K., Wang, Z., Yi, P., Chen, C., Han, Z., Lu, T., Huang, B., Jiang, J.: Decomposition makes better rain removal: an improved attention-guided deraining network. IEEE Trans. Circuits Syst. Video Technol. 31(10), 3981–3995 (2021)

    Article  Google Scholar 

  39. Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., Jiang, J.: Multi-scale progressive fusion network for single image deraining. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8346–8355 (2020)

  40. Zheng, S., Lu, C., Wu, Y., Gupta, G.: SAPNet: Segmentation-aware progressive network for perceptual contrastive deraining. In: IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 52–62 (2022)

  41. Li, B., Liu, X., Hu, P., Wu, Z., Lv, J., Peng, X.: All-in-one image restoration for unknown corruption. In: IEEE Conference on Computer Vision and Pattern Recognition (2022)

  42. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., Yan, S.: Deep joint rain detection and removal from a single image. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1357–1366 (2017)

Download references

Author information

Authors and Affiliations

  1. School of Mathematics and Computer Sciences, Nanchang University, Nanchang, 330031, China

    Jibin Deng

  2. School of Information Engineering, Nanchang University, Nanchang, 330031, China

    Chaohua Hu

Authors
  1. Jibin Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chaohua Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jibin Deng.

Ethics declarations

Code

The code of MSDNet is accessible at https://github.com/WeLearn1314/MSDNet.

Conflict of interest

The author has no conflicts of interest and no financial incomes that may influence it.

Ethical approval

Not applicable.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, J., Hu, C. A new multi-scale CNN with pixel-wise attention for image denoising. SIViP 18, 2733–2741 (2024). https://doi.org/10.1007/s11760-023-02944-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-023-02944-x

Keywords

Search

Navigation