Skip to main content
SpringerLink
Log in

MACFNet: multi-attention complementary fusion network for image denoising

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Recent years, thanks to the prosperous development of deep convolutional neural network, image denoising task has achieved unprecedented achievements. However, previous researches have difficulties in keeping the balance between noise removing and textual details preserving, even bringing the negative effect, such as local blurring. To overcome these weaknesses, in this paper, we propose an innovative multi-attention complementary fusion network (MACFNet) to restore delicate texture details while eliminating noise to the greatest extent. To be specific, our proposed MACFNet mainly composes of several multi-attention complementary fusion modules (MACFMs). Firstly, we use feature extraction block (FEB) to extract basic features.Then, we use spatial attention (SA), channel attention (CA) and patch attention (PA) three different kinds of attention mechanisms to extract spatial-dimensional, channel-dimensional and patch-dimensional attention aware features, respectively. In addition, we attempt to integrate three attention mechanisms in an effective way. Instead of directly concatenate, we design a subtle complementary fusion block (CFB), which is skilled in incorporating three sub-branches characteristics adaptively. Extensive experiments are carried out on gray-scale image denoising, color image denoising and real noisy image denoising. The quantitative results (PSNR) and visual effects all prove that our proposed network achieves great performance over some state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Yu X, Ye X, Zhang S (2022) Floating pollutant image target extraction algorithm based on immune extremum region. Digit Sig Process 123:103,442–103,458. https://doi.org/10.1016/j.dsp.2022.103442

    Article  Google Scholar 

  2. Yu X, Tian X (2022) A fault detection algorithm for pipeline insulation layer based on immune neural network. Int J Press Vessel Pip 196:104,611–104,625. https://doi.org/10.1016/j.ijpvp.2022.104611

    Article  Google Scholar 

  3. Liu X, Zhu X, Li M, et al. (2020) Multiple kernel k k-means with incomplete kernels. IEEE Trans Pattern Anal Mach Intell 42(5):1191–1204. https://doi.org/10.1109/TPAMI.2019.2892416

    Article  Google Scholar 

  4. Chen T, Ma KK, Chen LH (1999) Tri-state median filter for image denoising. IEEE Trans Image Process 8(12):1834–1838. https://doi.org/10.1109/83.806630

    Article  Google Scholar 

  5. Chen J, Benesty J, Huang Y et al (2006) New insights into the noise reduction wiener filter. IEEE Trans Audio, Speech, Lang Process 14(4):1218–1234. https://doi.org/10.1109/TSA.2005.860851

    Article  Google Scholar 

  6. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Proceedings of the IEEE international conference on computer vision, pp 839–846

  7. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: Proceedings of the computer society conference on computer vision and pattern recognition, vol 2, pp 60–65. https://doi.org/10.1109/CVPR.2005.38

  8. Dabov K, Foi A, Katkovnik V et al (2007) Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095. https://doi.org/10.1109/TIP.2007.901238

    Article  MathSciNet  Google Scholar 

  9. Fedorov V, Ballester C (2017) Affine non-local means image denoising. IEEE Trans Image Process 26(5):2137–2148. https://doi.org/10.1109/TIP.2017.2681421

    Article  MathSciNet  MATH  Google Scholar 

  10. Li X, Zhou Y, Zhang J et al (2020) Multipatch unbiased distance non-local adaptive means with wavelet shrinkage. IEEE Trans Image Process 29:157–169. https://doi.org/10.1109/TIP.2019.2928644

    Article  MathSciNet  MATH  Google Scholar 

  11. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745. https://doi.org/10.1109/TIP.2006.881969

    Article  MathSciNet  Google Scholar 

  12. Ren C, He X, Wang C, et al. (2021) Adaptive consistency prior based deep network for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 8596–8606

  13. Mustafi A, Ghorai S (2013) A novel blind source separation technique using fractional fourier transform for denoising medical images. Optik 124(3):265–271. https://doi.org/10.1016/j.ijleo.2011.11.052

    Article  Google Scholar 

  14. Wang X (2006) Moving window-based double haar wavelet transform for image processing. IEEE Trans Image Process 15(9):2771–2779. https://doi.org/10.1109/TIP.2006.877316

    Article  Google Scholar 

  15. Jain V, Seung HS (2008) Natural image denoising with convolutional networks. In: Proceedings of the international conference on neural information processing systems, pp 769–776

  16. Zamir SW, Arora A, Khan S et al (2021) Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 14,821–14,831

  17. Zhang K, Zuo W, Chen Y et al (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155. https://doi.org/10.1109/TIP.2017.2662206

    Article  MathSciNet  MATH  Google Scholar 

  18. Zhang K, Zuo W, Zhang L (2018) Ffdnet: toward a fast and flexible solution for cnn-based image denoising. IEEE Trans Image Process 27(9):4608–4622. https://doi.org/10.1109/TIP.2018.2839891

    Article  MathSciNet  Google Scholar 

  19. Yang D, Sun J (2018) Bm3d-net: a convolutional neural network for transform-domain collaborative filtering. IEEE Sig Process Lett 25(1):55–59. https://doi.org/10.1109/LSP.2017.2768660

    Article  Google Scholar 

  20. Liu P, Zhang H, Zhang K et al (2018) Multi-level wavelet-cnn for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 773–782

  21. Plötz T, Roth S (2018) Neural nearest neighbors networks. In: Proceedings of the international conference on neural information processing systems, pp 1095–1106

  22. Liu D, Wen B, Fan Y et al (2018) Non-local recurrent network for image restoration. In: Proceedings of the international conference on neural information processing systems, pp 1680–1689

  23. Zhang Y, Li K, Zhong B et al (2019) Residual non-local attention networks for image restoration. In: Proceedings of the international conference on learning representations, pp 1–18

  24. Wang Y, Song X, Chen K (2021) Channel and space attention neural network for image denoising. IEEE Sig Process Lett 28:424–428. https://doi.org/10.1109/LSP.2021.3057544

    Article  Google Scholar 

  25. Zhang Y, Li K, Li K et al (2021) Accurate and fast image denoising via attention guided scaling. IEEE Trans Image Process 30:6255–6265. https://doi.org/10.1109/TIP.2021.3093396

    Article  Google Scholar 

  26. Zamir SW, Arora A, Khan S et al (2020) Learning enriched features for real image restoration and enhancement. In: Proceedings of the European conference on computer vision, pp 492–511

  27. Wang T, Sun M, Hu K (2017) Dilated deep residual network for image denoising. In: Proceedings of the IEEE international conference on tools with artificial intelligence, pp 1272–1279

  28. Zou W, Jiang M, Zhang Y et al (2021) Sdwnet: a straight dilated network with wavelet transformation for image deblurring. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1895–1904

  29. Yue Z, Zhao Q, Zhang L et al (2020) Dual adversarial network: Toward real-world noise removal and noise generation. In: Proceedings of the European conference on computer vision, pp 41–58

  30. Kim Y, Soh JW, Park GY et al (2020) Transfer learning from synthetic to real-noise denoising with adaptive instance normalization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 3482–3492

  31. Zhang Y, Tian Y, Kong Y et al (2021) Residual dense network for image restoration. IEEE Trans Pattern Anal Mach Intell 43(7):2480–2495. https://doi.org/10.1109/TPAMI.2020.2968521

    Article  Google Scholar 

  32. Zhang Y, Li K, Wang L et al (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision, pp 286–301

  33. Liang J, Cao J, Sun G et al (2021) Swinir: image restoration using swin transformer. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 1833–1844

  34. Zheng H, Yong H, Zhang L (2021) Deep convolutional dictionary learning for image denoising. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 630–641

  35. Wang F, Jiang M, Qian C et al (2017) Residual attention network for image classification. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3156–3164

  36. Sinha A, Dolz J (2021) Multi-scale self-guided attention for medical image segmentation. IEEE J Biomed Health Inform 25(1):121–130. https://doi.org/10.1109/JBHI.2020.2986926

    Article  Google Scholar 

  37. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 7132–7141

  38. Zhao H, Zhang Y, Liu S et al (2018) Psanet: point-wise spatial attention network for scene parsing. In: Proceedings of the European conference on computer vision, pp 267–283

  39. Anwar S, Barnes N (2019) Real image denoising with feature attention. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3155–3164

  40. Mou C, Zhang J, Fan X et al (2022) Cola-net: collaborative attention network for image restoration. IEEE Trans Multimed 24:1366–1377. https://doi.org/10.1109/TMM.2021.3063916

    Article  Google Scholar 

  41. Agustsson E, Timofte R (2017) Ntire 2017 challenge on single image super-resolution: dataset and study. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 126–135

  42. Mairal J, Bach F, Ponce J et al (2009) Non-local sparse models for image restoration. In: Proceedings of the IEEE international conference on computer vision, pp 2272–2279

  43. Martin D, Fowlkes C, Tal D et al (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the IEEE international conference on computer vision, pp 416–423

  44. Franzen R (1999) Kodak lossless true color image suite. vol. 4 no. 2. http://r0k.us/graphics/kodak/. Accessed 15 Nov 1999

  45. Huang JB, Singh A, Ahuja N (2015) Single image super-resolution from transformed self-exemplars. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5197–5206

  46. Lebrun M, Colom M, Morel JM (2015) The noise clinic: a blind image denoising algorithm. Image Process Line 5:1–54. https://doi.org/10.5201/ipol.2015.125

    Article  MATH  Google Scholar 

  47. Gu S, Zhang L, Zuo W et al (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869

  48. Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: Proceedings of the IEEE international conference on computer vision, pp 479–486

  49. Harmeling S, Schuler C, Burger H (2012) Image denoising: can plain neural networks compete with bm3d?. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2392–2399

  50. Chen Y, Pock T (2017) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell 39(6):1256–1272. https://doi.org/10.1109/TPAMI.2016.2596743

    Article  Google Scholar 

  51. Zhang K, Zuo W, Gu S et al (2017) Learning deep cnn denoiser prior for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3929–3938

  52. Tian C, Xu Y, Fei L et al (2019) Enhanced cnn for image denoising. CAAI Trans Intell Technol 4(1):17–23. https://doi.org/10.1049/trit.2018.1054

    Article  Google Scholar 

  53. Zhang L, Li Y, Wang P et al (2019) A separation–aggregation network for image denoising. Appl Soft Comput 83:105,603. https://doi.org/10.1016/j.asoc.2019.105603

    Article  Google Scholar 

  54. Shi M, Fan L, Li X et al (2022) A competent image denoising method based on structural information extraction. Vis Comput:1–17. https://doi.org/10.1007/s00371-022-02491-5

  55. Zhang Q, Xiao J, Tian C et al (2022) A robust deformed convolutional neural network (cnn) for image denoising. In: Proceedings of the CAAI transactions on intelligence technology, pp 1–12. https://doi.org/10.1049/cit2.12110

  56. Tian C, Xu Y, Zuo W et al (2021) Designing and training of a dual cnn for image denoising. Knowl-Based Syst 226:106,949. https://doi.org/10.1016/j.knosys.2021.106949

    Article  Google Scholar 

  57. Xu M, Xie X (2022) Nfcnn: toward a noise fusion convolutional neural network for image denoising. SIViP 16(1):175–183. https://doi.org/10.1007/s11760-021-01965-8

    Article  Google Scholar 

  58. Xu J, Deng X, Xu M (2022) Revisiting convolutional sparse coding for image denoising: from a multi-scale perspective. IEEE Sig Process Lett 29:1202–1206. https://doi.org/10.1109/LSP.2022.3175096

    Article  Google Scholar 

  59. Mao XJ, Shen C, Yang YB (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Proceedings of the 30th international conference on neural information processing systems, pp 2810–2818

  60. Tai Y, Yang J, Liu X et al (2017) Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE international conference on computer vision, pp 4539–4547

  61. Zhu X, Lyu S, Wang X et al (2021) Tph-yolov5: improved yolov5 based on transformer prediction head for object detection on drone-captured scenarios. In: Proceedings of the IEEE/CVF international conference on computer vision workshops, pp 2778–2788

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, 333 Longteng Road, Shanghai, 201620, China

    Jiaolong Yu, Juan Zhang & Yongbin Gao

Authors
  1. Jiaolong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yongbin Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Juan Zhang.

Ethics declarations

Conflict of Interests

The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, J., Zhang, J. & Gao, Y. MACFNet: multi-attention complementary fusion network for image denoising. Appl Intell 53, 16747–16761 (2023). https://doi.org/10.1007/s10489-022-04313-6

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04313-6

Keywords

Search

Navigation