Skip to main content
SpringerLink
Log in

FEMRNet: Feature-enhanced multi-scale residual network for image denoising

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep convolutional neural networks (DCNN) have attracted considerable interest in image denoising because of their excellent learning capacity. However, most of the existing methods cannot fully extract and utilize the fine features during denoising, resulting in insufficient detailed information extracted and limited model expression ability, especially in complex denoising tasks. Inspired by the above challenges, in this paper, a feature-enhanced multi-scale residual network (FEMRNet) is proposed, mainly including an enhanced feature extraction block (EFEB), a multi-scale residual backbone (MSRB), a detail information recovery block (DIRB) and a merge reconstruction block (MRB). Specifically, the EFEB can increase the receptive field through dilated convolution with different expansion factors, and multi-scale convolution can further enhance the feature. The MSRB integrates global and local feature information through residual denoising blocks and skip connections to enhance the inferencing ability of denoising models. The DIRB is used to finely extract the information in the image, and combine the timing information by convGRU to restore the image details. Finally, MRB is designed to construct a clean image by subtracting the fused noise mapping obtained from MSRB and DIRB with a given noisy image. Additionally, extensive experiments are implemented on commonly-used denoising benchmarks. Comparison experiments with state-of-the-art methods and ablation experiments show that our method achieves promising performance in denoising tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. He W, Zhang H, Shen H, Zhang L (2018) Hyperspectral image denoising using local low-rank matrix recovery and global spatial–spectral total variation. IEEE J Sel Top Appl Earth Obs Remote Sens 11(3):713–729

    Google Scholar 

  2. Zhao W, Lu H (2017) Medical image fusion and denoising with alternating sequential filter and adaptive fractional order total variation. IEEE Trans Instrum Meas 66(9):2283–2294

    Google Scholar 

  3. Liu N, Wang J, Gao J, Chang S, Lou Y (2022) Similarity-informed self-learning and its application on seismic image denoising. IEEE Trans Geosci Remote Sens 60:1–13

    Google Scholar 

  4. Shi Q, Tang X, Yang T, Liu R, Zhang L (2021) Hyperspectral image denoising using a 3-D attention denoising network. IEEE Trans Geosci Remote Sens 59(12):10348–10363

    Google Scholar 

  5. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), vol 2. IEEE, pp 60–65

    Google Scholar 

  6. Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869

    Google Scholar 

  7. Dong W, Zhang L, Shi G, Li X (2012) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    MathSciNet  MATH  Google Scholar 

  8. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    MathSciNet  Google Scholar 

  9. Aharon M, Elad M, Bruckstein A (2006) K-SVD: An algorithm for designingovercomplete dictionaries for sparse representation. IEEE Trans. Signal Process 54(11):4311–4322

    MATH  Google Scholar 

  10. Vardan P, Yaniv R, Jeremias S, Michael E (2018) Theoretical foundations of deep learning via sparse representations: a multilayer sparse model and its connection to convolutional neural networks. IEEE Signal Process Mag 35(4):72–89

    Google Scholar 

  11. Izadi S, Sutton D, Hamarneh G (2023) Image denoising in the deep learning era. Artif Intell Rev 56:5929–5974. https://doi.org/10.1007/s10462-022-10305-2

    Article  Google Scholar 

  12. Tian C, Fei L, Zheng W, Xu Y, Zuo W, Lin CW (2020) Deep learning on image denoising: an overview. Neural Netw 131:251–275

    MATH  Google Scholar 

  13. Ilesanmi AE, Ilesanmi TO (2021) Methods for image denoising using convolutional neural network: a review. Complex Intell Syst 7(5):2179–2198

    Google Scholar 

  14. Jain V, Murray JF, Roth F, Turaga S, Zhigulin V, Briggman KL, Seung HS (2007) Supervised learning of image restoration with convolutional networks. In: 2007 IEEE 11th international conference on computer vision. IEEE, pp 1–8

    Google Scholar 

  15. Burger HC, Schuler CJ, Harmeling S (2012) Image denoising: can plain neural networks compete with bm3d? In: 2012 IEEE conference on computer vision and pattern recognition. IEEE, pp 2392–2399

    Google Scholar 

  16. Schmidt U, Roth S (2014) Shrinkage fields for effective image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2774–2781

    Google Scholar 

  17. Chen Y, Pock T (2016) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell 39(6):1256–1272

    Google Scholar 

  18. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155

    MathSciNet  MATH  Google Scholar 

  19. Zhang K, Zuo W, Zhang L (2018) FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans Image Process 27(9):4608–4622

    MathSciNet  Google Scholar 

  20. Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3929–3938

    Google Scholar 

  21. Tian C, Xu Y, Fei L, Wang J, Wen J, Luo N (2019) Enhanced cnn for image denoising. CAAI Trans Intell Technol 4(1):17–23

    Google Scholar 

  22. Sheng J, Lv G, Wang Z, Feng Q (2022) SRNet: sparse representation-based network for image denoising. Digit Signal Process 130:103702

    Google Scholar 

  23. Zheng M, Zhi K, Zeng J, Tian C, You L (2022) A hybrid CNN for image denoising. J Artif Intell Technol 2(3):93–99

    Google Scholar 

  24. Xu S, Zhang J, Wang J, Sun K, Zhang C, Liu J, Hu J (2022) A model-driven network for guided image denoising. Inf Fusion 85:60–71. https://doi.org/10.1016/j.inffus.2022.03.006

    Article  Google Scholar 

  25. Liu G, Dang M, Liu J, Xiang R, Tian Y, Luo N (2022) True wide convolutional neural network for image denoising. Inf Sci 610:171–184. https://doi.org/10.1016/j.ins.2022.07.122

    Article  Google Scholar 

  26. Su J, Xu B, Yin H (2022) A survey of deep learning approaches to image restoration. Neurocomputing 487:46–65. https://doi.org/10.1016/j.neucom.2022.02.046

    Article  Google Scholar 

  27. Lv T, Pan X, Zhu Y, Li L (2021) Unsupervised medical images denoising via graph attention dual adversarial network. Appl Intell 51(6):4094–4105. https://doi.org/10.1007/s10489-020-02016-4

    Article  Google Scholar 

  28. Wang X, Wu K, Zhang Y, Xiao Y, Xu P (2023) A Gan-based denoising method for chinese stele and rubbing calligraphic image. Vis Comput 39(4):1351–1362

    Google Scholar 

  29. Wang Y, Chang D, Zhao Y (2021) A new blind image denoising method based on asymmetric generative adversarial network. IET Image Process 15(6):1260–1272

    Google Scholar 

  30. Chung H, Lee ES, Ye JC (2022) MR image denoising and super-resolution usingregularized reverse diffusion. IEEE Trans. Med. Imaging 42(4):922–934. https://doi.org/10.1109/TMI.2022.3220681

    Article  Google Scholar 

  31. Lugmayr A, Danelljan M, Romero A, Yu F, Timofte R, Van Gool L (2022) Repaint: Inpainting using denoising diffusion probabilistic models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 11461–11471

    Google Scholar 

  32. Huang T, Li S, Jia X, Lu H, Liu J (2022) Neighbor2neighbor: a self-supervised framework for deep image denoising. IEEE Trans Image Process 31:4023–4038. https://doi.org/10.1109/TIP.2022.3176533

    Article  Google Scholar 

  33. Wang Z, Liu J, Li G, Han H (2022) Blind2unblind: self-supervised image denoising with visible blind spots. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2027–2036

    Google Scholar 

  34. Zhang Q, Xiao J, Tian C et al (2022) A robust deformed convolutional neural network (cnn) for image denoising. In: Proceedings of the CAAI transactions on intelligence technology, pp 1–12. https://doi.org/10.1049/cit2.12110

    Chapter  Google Scholar 

  35. Herbreteau S, Kervrann C (2022) DCT2net: an interpretable shallow CNN for image denoising. IEEE Trans Image Process 31:4292–4305. https://doi.org/10.1109/TIP.2022.3181488

    Article  Google Scholar 

  36. Tian C, Zheng M, Zuo W, Zhang B, Zhang Y, Zhang D (2023) Multi-stage image denoising with the wavelet transform. Pattern Recogn 134:109050

    Google Scholar 

  37. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE international conference on computer vision, pp 4539–4547

    Google Scholar 

  38. Mao X, Shen C, Yang YB (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in neural information processing systems, pp 2802–2810

    Google Scholar 

  39. Chen H, Zhang Y, Kalra MK, Lin F, Chen Y, Liao P, Wang G (2017) Low-dose CT with a residual encoder-decoder convolutional neural network. IEEE Trans Med Imaging 36(12):2524–2535

    Google Scholar 

  40. Liu P, Zhang H, Zhang K, Lin L, Zuo W (2018) Multi-level wavelet-CNN for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 773–782

    Google Scholar 

  41. Schmidt C, Athar A, Mahadevan S, Leibe B (2022) D2Conv3D: dynamic dilated convolutions for object segmentation in videos. In: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp 1200–1209

    Google Scholar 

  42. Gao J, Gong M, Li X (2022) Congested crowd instance localization with dilated convolutional swin transformer. Neurocomputing 513:94–103

    Google Scholar 

  43. Jiang W, Liu M, Peng Y, Wu L, Wang Y (2020) HDCB-net: a neural network with the hybrid dilated convolution for pixel-level crack detection on concrete bridges. IEEE Trans Industr Inform 17(8):5485–5494

    Google Scholar 

  44. Li Y, Li X, Xiao C, Li H, Zhang W (2021) EACNet: enhanced asymmetric convolution for real-time semantic segmentation. IEEE Signal Process Lett 28:234–238

    Google Scholar 

  45. Tian C, Xu Y, Li Z, Zuo W, Fei L, Liu H (2020) Attention-guided cnn for image denoising. Neural Netw 124:117–129. https://doi.org/10.1016/j.neunet.2019.12.024

    Article  Google Scholar 

  46. Tian C, Xu Y, Zuo W, Du B, Lin CW, Zhang D (2021) Designing and training of a dual cnn for image denoising. Knowled-Based Syst 226:106–949. https://doi.org/10.1016/j.knosys.2021.106949

    Article  Google Scholar 

  47. Chen YW, Yang HK, Chiu CC, Lee CY (2022) S2F2: single-stage flow forecasting for future multiple trajectories prediction. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2536–2539

    Google Scholar 

  48. Ye R, Li X, Ye Y, Zhang B (2022) DynamicNet: a time-variant ODE network for multi-step wind speed prediction. Neural Netw 152:118–139

    Google Scholar 

  49. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings eighth IEEE international conference on computer vision, vol 2. IEEE, pp 416–423

    Google Scholar 

  50. Ma K, Duanmu Z, Wu Q, Wang Z, Yong H, Li H, Zhang L (2016) Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016

    MathSciNet  MATH  Google Scholar 

  51. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops pp 126–135

    Google Scholar 

  52. Guo S, Yan Z, Zhang K, Zuo W, Zhang L (2019) Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1712–1722

  53. Roth S, Black MJ (2005) Fields of experts: a framework for learning image priors. In: 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR’05), Vol 2. Citeseer, pp 860–867

    Google Scholar 

  54. Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration. In: 2009 IEEE 12th international conference on computer vision. IEEE, pp 2272–2279

    Google Scholar 

  55. Franzen R (1999) Kodak lossless true color image suite vol 4, http://r0k.us/graphics/kodak

  56. Zhang L, Wu X, Buades A, Li X (2011) Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J Electron imaging 20(2):023016

    Google Scholar 

  57. Nam S, Hwang Y, Matsushita Y, Joo Kim S (2016) A holistic approach to cross-channel image noise modeling and its application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1683–1691

    Google Scholar 

  58. Huynh-Thu Q, Ghanbari M (2008) Scope of validity of psnr in image/video quality assessment. Electron Lett 44(13):800–801

    Google Scholar 

  59. Hore A, Ziou D (2010) Image quality metrics: PSNR vs. SSIM. In: 2010 20th international conference on pattern recognition. IEEE, pp 2366–2369

    Google Scholar 

  60. Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: 2011 International conference on computer vision, pp 479–486

    Google Scholar 

  61. Shi M, Fan L, Li X et al (2023) A competent image denoising method based on structuralinformation extraction. Vis Comput 39:2407–2423. https://doi.org/10.1007/s00371-022-02491-5

    Article  Google Scholar 

  62. Xu J, Deng X, Xu M (2022) Revisiting convolutional sparse coding for image denoising: from a multi-scale perspective. IEEE Sig Process Lett 29:1202–1206. https://doi.org/10.1109/LSP.2022.3175096

    Article  Google Scholar 

  63. Luo E, Chan SH, Nguyen TQ (2015) Adaptive image denoising by targeted databases. IEEE Trans Image Process 24(7):2167–2181

    MathSciNet  MATH  Google Scholar 

  64. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  65. Reyes O, Altalhi AH, Ventura S (2018) Statistical comparisons of active learning strategies over multiple datasets. Knowl-Based Syst 145:274–288

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundations of China (no.62276265, no. 61976216, no.62206296, and no.62206297) and the Fundamental Research Funds for the Central Universities (no.2022QN1096).

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, China University of Mining and Technology, Xuzhou, 221116, China

    Xiao Xu, Qidong Wang, Lili Guo, Jian Zhang & Shifei Ding

  2. Xuzhou First People’s Hospital, Xuzhou, 221116, China

    Xiao Xu

Authors
  1. Xiao Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qidong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lili Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jian Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shifei Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shifei Ding.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, X., Wang, Q., Guo, L. et al. FEMRNet: Feature-enhanced multi-scale residual network for image denoising. Appl Intell 53, 26027–26049 (2023). https://doi.org/10.1007/s10489-023-04895-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04895-9

Keywords

Search

Navigation