Skip to main content
Springer Nature Link
Log in

Multi-scale information fusion generative adversarial network for real-world noisy image denoising

  • Research
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

Image denoising is crucial for enhancing image quality, improving visual effects, and boosting the accuracy of image analysis and recognition. Most of the current image denoising methods perform superior on synthetic noise images, but their performance is limited on real-world noisy images since the types and distributions of real noise are often uncertain. To address this challenge, a multi-scale information fusion generative adversarial network method is proposed in this paper. Specifically, In this method, the generator is an end-to-end denoising network that consists of a novel encoder–decoder network branch and an improved residual network branch. The encoder–decoder branch extracts rich detailed and contextual information from images at different scales and utilizes a feature fusion method to aggregate multi-scale information, enhancing the feature representation performance of the network. The residual network further compensates for the compressed and lost information in the encoder stage. Additionally, to effectively aid the generator in accomplishing the denoising task, convolution kernels of various sizes are added to the discriminator to improve its image evaluation ability. Furthermore, the dual denoising loss function is presented to enhance the model’s capability in performing noise removal and image restoration. Experimental results show that the proposed method exhibits superior objective performance and visual quality than some state-of-the-art methods on three real-world datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The data and materials are available from the corresponding author on reasonable request.

References

  1. Zeng, N., Wu, P., Wang, Z., Li, H., Liu, W., Liu, X.: A small-sized object detection oriented multi-scale feature fusion approach with application to defect detection. IEEE Trans. Instrum. Meas. 71, 1–14 (2022)

    Google Scholar 

  2. Ning, X., Tian, W., Yu, Z., Li, W., Bai, X., Wang, Y.: Hcfnn: high-order coverage function neural network for image classification. Pattern Recognit. 131, 108873 (2022)

    Article  Google Scholar 

  3. Cheng, Z., Qu, A., He, X.: Contour-aware semantic segmentation network with spatial attention mechanism for medical image. The Vis. Comput. 38, 749–762 (2022)

    Article  Google Scholar 

  4. Buades, A., Coll, B., Morel, J.-M.: A non-local algorithm for image denoising. In: 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 60–65 (2005)

  5. Dabov, K., Foi, A., Katkovnik, V., Egiazarian, K.: Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans. Image Process. 16(8), 2080–2095 (2007)

    Article  MathSciNet  Google Scholar 

  6. Aharon, M., Elad, M., Bruckstein, A.: K-svd: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Signal Process. 54(11), 4311–4322 (2006)

    Article  Google Scholar 

  7. Gu, S., Zhang, L., Zuo, W., Feng, X.: Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2862–2869 (2014)

  8. Pan, Y., Ren, C., Wu, X., Huang, J., He, X.: Real image denoising via guided residual estimation and noise correction. IEEE Trans. Circuits Syst. Video Technol. 33(4), 1994–2000 (2022)

    Article  Google Scholar 

  9. Jain, V., Seung, S.: Natural image denoising with convolutional networks. Adv. Neural Inf. Process. Syst. 21 (2008)

  10. Zhang, K., Zuo, W., Chen, Y., Meng, D., Zhang, L.: Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans. Image Process. 26(7), 3142–3155 (2017)

    Article  MathSciNet  Google Scholar 

  11. Zhang, K., Zuo, W., Zhang, L.: Ffdnet: toward a fast and flexible solution for cnn-based image denoising. IEEE Trans. Image Process. 27(9), 4608–4622 (2018)

    Article  MathSciNet  Google Scholar 

  12. Guo, S., Yan, Z., Zhang, K., Zuo, W., Zhang, L.: Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1712–1722 (2019)

  13. Anwar, S., Barnes, N.: Real image denoising with feature attention. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 3155–3164 (2019)

  14. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H., Shao, L.: Learning enriched features for real image restoration and enhancement. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16, pp. 492–511 (2020)

  15. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., Yang, M.-H., Shao, L.: Multi-stage progressive image restoration. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14821–14831 (2021)

  16. Jiang, B., Lu, Y., Wang, J., Lu, G., Zhang, D.: Deep image denoising with adaptive priors. IEEE Trans. Circuits Syst. Video Technol. 32(8), 5124–5136 (2022)

    Article  Google Scholar 

  17. Zhou, L., Zhou, D., Yang, H., Yang, S.: Multi-scale network toward real-world image denoising. Int. J. Mach. Learn. Cybern. 14(4), 1205–1216 (2023)

    Article  Google Scholar 

  18. Jia, X., Peng, Y., Ge, B., Li, J., Liu, S., Wang, W.: A multi-scale dilated residual convolution network for image denoising. Neural Process. Lett. 55(2), 1231–1246 (2023)

    Article  Google Scholar 

  19. Zhou, L., Zhou, D., Yang, H., Yang, S.: Two-subnet network for real-world image denoising. Multimed. Tools Appl., 1–17 (2023)

  20. Zuo, Y., Yao, W., Zeng, Y., Xie, J., Fang, Y., Huang, Y., Jiang, W.: Cfnet: conditional filter learning with dynamic noise estimation for real image denoising. Knowl.-Based Syst. 284, 111320 (2024)

    Article  Google Scholar 

  21. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Adv. Neural Inf. Process. Syst. 27 (2014)

  22. Zhao, J., Lee, F., Hu, C., Yu, H., Chen, Q.: Lda-gan: lightweight domain-attention gan for unpaired image-to-image translation. Neurocomputing 506, 355–368 (2022)

    Article  Google Scholar 

  23. Chen, Y., Xia, R., Yang, K., Zou, K.: Gcam: lightweight image inpainting via group convolution and attention mechanism. Int. J. Mach. Learn. Cybern. 1–11 (2023)

  24. Chen, Y., Xia, R., Yang, K., Zou, K.: Dargs: image inpainting algorithm via deep attention residuals group and semantics. J. King Saud Univ.-Comput. Inf. Sci. 35(6), 101567 (2023)

    Google Scholar 

  25. Chen, Y., Xia, R., Yang, K., Zou, K.: Mfmam: image inpainting via multi-scale feature module with attention module. Comput. Vis. Image Underst. 238, 103883 (2024)

    Article  Google Scholar 

  26. Arjovsky, M., Chintala, S., Bottou, L.: Wasserstein generative adversarial networks. In: International Conference on Machine Learning, pp. 214–223 (2017)

  27. Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., Courville, A.C.: Improved training of wasserstein gans. Adv. Neural Inf. Process. Syst. 30 (2017)

  28. Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3155–3164 (2018)

  29. Lin, K., Li, T.H., Liu, S., Li, G.: Real photographs denoising with noise domain adaptation and attentive generative adversarial network. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 1717–1721 (2019)

  30. Zhu, S., Xu, G., Cheng, Y., Han, X., Wang, Z.: Bdgan: Image blind denoising using generative adversarial networks. In: Pattern Recognition and Computer Vision: Second Chinese Conference, PRCV 2019, Xi’an, China, November 8–11, 2019, Proceedings, Part II 2, pp. 241–252 (2019)

  31. Kim, D.-W., Ryun Chung, J., Jung, S.-W.: Grdn: Grouped residual dense network for real image denoising and gan-based real-world noise modeling. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 2086–2094 (2019)

  32. Yue, Z., Zhao, Q., Zhang, L., Meng, D.: Dual adversarial network: Toward real-world noise removal and noise generation. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part X 16, pp. 41–58 (2020)

  33. Lyu, Q., Guo, M., Pei, Z.: Degan: mixed noise removal via generative adversarial networks. Appl. Soft Comput. 95, 106478 (2020)

    Article  Google Scholar 

  34. Vo, D.M., Nguyen, D.M., Le, T.P., Lee, S.-W.: Hi-gan: a hierarchical generative adversarial network for blind denoising of real photographs. Inf. Sci. 570, 225–240 (2021)

    Article  MathSciNet  Google Scholar 

  35. Zhao, S., Lin, S., Cheng, X., Zhou, K., Zhang, M., Wang, H.: Dual-gan complementary learning for real-world image denoising. IEEE Sens. J. 24(1), 355–366 (2024)

    Article  Google Scholar 

  36. Song, Y., Zhu, Y., Du, X.: Grouped multi-scale network for real-world image denoising. IEEE Signal Process. Lett. 27, 2124–2128 (2020)

    Article  Google Scholar 

  37. Wang, Y., Wang, G., Chen, C., Pan, Z.: Multi-scale dilated convolution of convolutional neural network for image denoising. Multimed. Tools Appl. 78, 19945–19960 (2019)

    Article  Google Scholar 

  38. Yu, X., Fu, Z., Ge, C.: A multi-scale generative adversarial network for real-world image denoising. Signal Image Video Process. 16, 257–264 (2022)

    Article  Google Scholar 

  39. Wang, Z., Wang, L., Duan, S., Li, Y.: An image denoising method based on deep residual gan. In: Journal of Physics: Conference Series, vol. 1550, p. 032127 (2020)

  40. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  41. Wu, W., Lv, G., Duan, Y., Liang, P., Zhang, Y., Xia, Y.: Dcanet: Dual convolutional neural network with attention for image blind denoising. arXiv preprint arXiv:2304.01498 (2023)

  42. Isola, P., Zhu, J.-Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

  43. Johnson, J., Alahi, A., Fei-Fei, L.: Perceptual losses for real-time style transfer and super-resolution. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pp. 694–711 (2016)

  44. Rubin, L.: Nonlinenr total variation based noise removal algorithms. Phys. D: Nonlinear Phenom. 60, 259–265 (1992)

    Article  Google Scholar 

  45. Li, R., Pan, J., Li, Z., Tang, J.: Single image dehazing via conditional generative adversarial network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8202–8211 (2018)

  46. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015)

  47. Chen, Y., Xia, R., Yang, K., Zou, K.: Micu: image super-resolution via multi-level information compensation and u-net. Expert Syst. Appl. 245, 123111 (2024)

    Article  Google Scholar 

  48. Chen, Y., Xia, R., Yang, K., Zou, K.: Dnnam: image inpainting algorithm via deep neural networks and attention mechanism. Appl. Soft Comput. 154, 111392 (2024)

    Article  Google Scholar 

  49. Odena, A., Dumoulin, V., Olah, C.: Deconvolution and checkerboard artifacts. Distill 1(10), 3 (2016)

    Article  Google Scholar 

  50. He, K., Zhang, X., Ren, S., Sun, J.: Identity mappings in deep residual networks. In: Computer Vision–ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11–14, 2016, Proceedings, Part IV 14, pp. 630–645 (2016)

  51. Lai, W.-S., Huang, J.-B., Ahuja, N., Yang, M.-H.: Deep laplacian pyramid networks for fast and accurate super-resolution. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 624–632 (2017)

  52. Seif, G., Androutsos, D.: Edge-based loss function for single image super-resolution. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 1468–1472 (2018)

  53. Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., Jiang, J.: Multi-scale progressive fusion network for single image deraining. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8346–8355 (2020)

  54. Kamgar-Parsi, B., Rosenfeld, A.: Optimally isotropic laplacian operator. IEEE Trans. Image Process. 8(10), 1467–1472 (1999)

    Article  MathSciNet  Google Scholar 

  55. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)

  56. Girshick, R.: Fast r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1440–1448 (2015)

  57. Abdelhamed, A., Lin, S., Brown, M.S.: A high-quality denoising dataset for smartphone cameras. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1692–1700 (2018)

  58. Plotz, T., Roth, S.: Benchmarking denoising algorithms with real photographs. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1586–1595 (2017)

  59. Xu, J., Li, H., Liang, Z., Zhang, D., Zhang, L.: Real-world noisy image denoising: A new benchmark. arXiv preprint arXiv:1804.02603 (2018)

  60. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the reviewers for all of their careful, constructive and insightful comments in relation to this work.

Funding

No Financial support.

Author information

Authors and Affiliations

  1. School of Science, Chongqing University of Posts and Telecommunications, Chongqing, 400065, People’s Republic of China

    Xuegang Hu & Wei Zhao

Authors
  1. Xuegang Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Zhao.

Ethics declarations

Conflict of interest

The authors declare that they have no Conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, X., Zhao, W. Multi-scale information fusion generative adversarial network for real-world noisy image denoising. Machine Vision and Applications 35, 81 (2024). https://doi.org/10.1007/s00138-024-01563-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00138-024-01563-x

Keywords