Skip to main content
SpringerLink
Log in

Learning image blind denoisers without explicit noise modeling

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image blind denoising aims at removing the unknown noise from given images to improve the image’s visual quality. Current blind denoisers can be categorized into two classes, i.e., traditional and deep learning-based methods. Generally, deep learning-based methods are likely to outperform the traditional methods when the model of image noise is within the training noise distribution. However, when the image noise is unseen during the optimization process, traditional methods usually achieve better denoising performance. To address the generalization problem of deep learning-based denoiser, our paper proposed a general yet concise image blind denoising framework that can be applied to existing deep learning-based denoisers. Specifically, given noisy images, our method first extracts high-frequency signals from the images and synthesizes noise from the extracted signals using several noise generation techniques. In contrast to the existing conventional approaches that explicitly model the noise, our proposed framework advances in generating noise directly from the noisy images without explicitly noise modeling. By applying the generated noise to clean images to construct noisy-clean paired images, the deep learning-based denoisers can be optimized using the constructed data with the similar noise of the given noisy images. After optimization, the denoisers are capable of removing noise from the given noisy images. Extensive experiments and comprehensive ablations on both synthetic noisy image and real-world image denoising tasks demonstrate the superiority of our proposed framework over all the compared approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

The authors declare that all data supporting the findings of this study are available within the article.

References

  1. ABSoft N (2022) Neat image. http://ni.neatvideo.com/home. Accessed 07 Sep 2018

  2. Ali A, Zhu Y, Chen Q, Yu J, Cai H (2019) Leveraging spatio-temporal patterns for predicting citywide traffic crowd flows using deep hybrid neural networks. In: 2019 IEEE 25th international conference on parallel and distributed systems (ICPADS). IEEE, pp 125–132

  3. Ali A, Zhu Y, Zakarya M (2021) Exploiting dynamic spatio-temporal correlations for citywide traffic flow prediction using attention based neural networks. Inf Sci 577:852–870

    Article  MathSciNet  Google Scholar 

  4. Ali A, Zhu Y, Zakarya M (2022) Exploiting dynamic spatio-temporal graph convolutional neural networks for citywide traffic flows prediction. Neural Netw 145:233–247

    Article  Google Scholar 

  5. Anayaa J, Barbua J (2014) Renoir-a dataset for real low-light noise image reduction. J Visual Commun Image Represent

  6. Association E et al (2010) Standard for characterization of image sensors and cameras. EMVA Standard, 1288

  7. Cao Q, Lin L, Shi Y, Liang X, Li G (2017) Attention-aware face hallucination via deep reinforcement learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 690–698

  8. Chen J, Chen J, Chao H, Yang M (2018) Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3155–3164

  9. Chen T, Lin L, Hui X, Chen R, Wu H (2020) Knowledge-guided multi-label few-shot learning for general image recognition. IEEE Trans Patt Anal Mach Intell

  10. Chen T, Pu T, Wu H, Xie Y, Liu L, Lin L (2021) Cross-domain facial expression recognition: a unified evaluation benchmark and adversarial graph learning. IEEE Trans Patt Anal Mach Intell

  11. Dabov K, Foi A, Katkovnik V, Egiazarian KO (2007) Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominance space. In: 2007 IEEE international conference on image processing, vol 1, pp I – 313–I –316

  12. Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    Article  MATH  MathSciNet  Google Scholar 

  13. Gonzalez RC, Woods RE (2001) Digital image processing, 2nd edn. Addison-Wesley Longman Publishing Co., Inc., Boston

    Google Scholar 

  14. Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  15. Granados M, Kim KI, Tompkin J, Theobalt C (2013) Automatic noise modeling for ghost-free hdr reconstruction. ACM Trans Graph 32:201:1–201:10

    Article  Google Scholar 

  16. Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869

  17. Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd international conference on international conference on machine learning., ICML’15, JMLR.org, vol 37, pp 448–456. http://dl.acm.org/citation.cfm?id=3045118.3045167

  18. Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. IEEE Conf Comput Vision Pattern Recognit:5967–5976

  19. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Int Conf Learn Represent

  20. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436

    Article  Google Scholar 

  21. Lebrun M, Buades A, Morel J-M (2013) A nonlocal bayesian image denoising algorithm. SIAM J Imaging Sci 6(3):1665–1688

    Article  MATH  MathSciNet  Google Scholar 

  22. Lebrun M, Colom M, Morel J-M (2015) The noise clinic: a blind image denoising algorithm. Image Processing Line 5:1–54

    Article  MATH  Google Scholar 

  23. Lebrun M, Colom M, Morel J-M (2015) Multiscale image blind denoising. IEEE Trans Image Process 24(10):3149–3161

    Article  MATH  MathSciNet  Google Scholar 

  24. Lefkimmiatis S (2017) Non-local color image denoising with convolutional neural networks. In: Proc. IEEE int. conf. computer vision and pattern recognition, pp 3587–3596

  25. Li H, Qin J, Yang Z, Wei P, Pan J, Lin L, Shi Y (2022) Real-world image super-resolution by exclusionary dual-learning, IEEE Trans Multimedia

  26. Lim B, Son S, Kim H, Nah S, Lee KM (2017) Enhanced deep residual networks for single image super-resolution. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 136–144

  27. Lin Y, Zhang S, Chen T, Lu Y, Li G, Shi Y (2022) Exploring negatives in contrastive learning for unpaired image-to-image translation, arXiv:2204.11018

  28. Liu C, Szeliski R, Kang SB, Zitnick CL, Freeman WT (2008) Automatic estimation and removal of noise from a single image. IEEE Trans Pattern Anal Mach Intell 30:299–314

    Article  Google Scholar 

  29. Liu X, Tanaka M, Okutomi M (2013) Single-image noise level estimation for blind denoising. IEEE transactions on image processing 22(12):5226–5237

    Article  Google Scholar 

  30. Ma K, Duanmu Z, Wu Q, Wang Z, Yong H, Li H, Zhang L (2017) Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016

    Article  MATH  MathSciNet  Google Scholar 

  31. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Computer vision, 2001. ICCV 2001. Proceedings. Eighth IEEE international conference on. IEEE, vol 2, pp 416–423

  32. Meng D, De La Torre F (2013) Robust matrix factorization with unknown noise. In: Proceedings of the IEEE international conference on computer vision, pp 1337–1344

  33. Nam S, Hwang Y, Matsushita Y, Joo Kim S (2016) A holistic approach to cross-channel image noise modeling and its application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1683–1691

  34. Plotz T, Roth S (2017) Benchmarking denoising algorithms with real photographs.. In: CVPR, vol, p 6

  35. Portilla J (2004) Full blind denoising through noise covariance estimation using gaussian scale mixtures in the wavelet domain. In: Image processing, 2004. ICIP’04. 2004 international conference on. IEEE, vol 2, pp 1217–1220

  36. Portilla J (2004) Blind non-white noise removal in images using gaussian scale mixtures in the wavelet domain. In: Benelux signal processing symposium

  37. Portilla J, Strela V, Wainwright MJ, Simoncelli EP (2003) Image denoising using scale mixtures of gaussians in the wavelet domain. IEEE Trans Image Process 12(11):1338–1351

    Article  MATH  MathSciNet  Google Scholar 

  38. Rabie T (2005) Robust estimation approach for blind denoising. IEEE Trans Image Process 14(11):1755–1765

    Article  Google Scholar 

  39. Santoro A, Bartunov S, Botvinick M, Wierstra D, Lillicrap T (2016) Meta-learning with memory-augmented neural networks. In: International conference on machine learning, pp 1842–1850

  40. Shi Y, Li G, Cao Q, Wang K, Lin L (2019) Face hallucination by attentive sequence optimization with reinforcement learning. IEEE Trans Pattern Anal Mach Intell 42(11):2809–2824

    Google Scholar 

  41. Shi Y, Qin J, Wei P, Ouyang W, Lin L (2019) Perceptual image enhancement by relativistic discriminant learning with cross-scale aggregated representation. IEEE Access 7:39660–39669

    Article  Google Scholar 

  42. Shi Y, Wang K, Chen C, Xu L, Lin L (2017) Structure-preserving image super-resolution via contextualized multitask learning. IEEE Trans Multimedia 19(12):2804–2815

    Article  Google Scholar 

  43. Shi Y, Wang K, Xu L, Lin L (2016) Local-and holistic-structure preserving image super resolution via deep joint component learning. In: 2016 IEEE international conference on multimedia and expo (ICME). IEEE, pp 1–6

  44. Shi Y, Zhang S, Zhou C, Liang X, Yang X, Lin L (2021) Gtae: graph transformer–based auto-encoders for linguistic-constrained text style transfer. ACM Trans Intell Syst Technol (TIST) 12(3):1–16

    Article  Google Scholar 

  45. Shi Y, Zhong H, Yang Z, Yang X, Lin L (2020) Ddet: dual-path dynamic enhancement network for real-world image super-resolution. IEEE Signal Process Lett 27:481–485

    Article  Google Scholar 

  46. Shocher A, Cohen N, Irani M (2018) Zero-shot super-resolution using deep internal learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3118–3126

  47. Tan J, Shi Y, Yang Z, Wen C, Lin L (2020) Unsupervised multi-view clustering by squeezing hybrid knowledge from cross view and each view. IEEE Trans Multimedia 23:2943–2956

    Article  Google Scholar 

  48. Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Computer vision, 1998. Sixth international conference on. IEEE, pp 839–846

  49. Xu J, Li H, Liang Z, Zhang D, Zhang L (2018) Real-world noisy image denoising: a new benchmark, CoRR arXiv:1804.02603

  50. Xu J, Zhang L, Zhang D (2018) External prior guided internal prior learning for real-world noisy image denoising. IEEE Trans Image Process 27(6):2996–3010

    Article  MATH  MathSciNet  Google Scholar 

  51. Xu J, Zhang L, Zhang D (2018) A trilateral weighted sparse coding scheme for real-world image denoising, CoRR arXiv:1807.04364

  52. Xu J, Zhang L, Zhang D, Feng X (2017) Multi-channel weighted nuclear norm minimization for real color image denoising. In: IEEE international conference on computer vision, vol 2

  53. Xu J, Zhang L, Zuo W, Zhang D, Feng X (2015) Patch group based nonlocal self-similarity prior learning for image denoising. In: Proceedings of the IEEE international conference on computer vision, pp 244–252

  54. Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. In: Proceedings of the European conference on computer vision (ECCV), pp 286–301

  55. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MATH  MathSciNet  Google Scholar 

  56. Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep cnn denoiser prior for image restoration. In: IEEE conference on computer vision and pattern recognition, vol 2

  57. Zhang K, Zuo W, Zhang L (2018) Ffdnet: toward a fast and flexible solution for cnn based image denoising. IEEE Trans mage Process

  58. Zhao Q, Meng D, Xu Z, Zuo W, Zhang L (2014) Robust principal component analysis with complex noise. In: International conference on machine learning, pp 55–63

  59. Zhu F, Chen G, Heng P-A (2016) From noise modeling to blind image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 420–429

  60. Zhu L, Fu C, Brown MS, Heng P (2017) A non-local low-rank framework for ultrasound speckle reduction. In: 2017 IEEE conference on computer vision and pattern recognition (CVPR), pp 493–501

  61. Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: Computer vision (ICCV), 2011 IEEE international conference on. IEEE, pp 479–486

Download references

Funding

This work is supported in part by National Natural Science Foundation of China (NSFC) (no. 62002069), the Science and Technology Project of Guangdong Province (no. 2021A1515011341, 2022A1515011835), China Postdoctoral Science Foundation funded project (Grant no. 2021M703687) and the Guangzhou Science and Technology Plan Project (no. 202002030386, 102020369).

Author information

Authors and Affiliations

  1. School of Automation Science and Engineering, South China University of Technology, Guangzhou, Guangdong, 510006, China

    Lin Nie & Wenxiong Kang

  2. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, Guangdong, 510006, China

    Junfan Lin & Liang Lin

  3. School of Information Engineering, Guangdong University of Technology, Guangzhou, Guangdong, 510006, China

    Yukai Shi

Authors
  1. Lin Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Junfan Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wenxiong Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yukai Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Junfan Lin.

Ethics declarations

Conflict of Interests

No potential conflict of interest was reported by the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nie, L., Lin, J., Kang, W. et al. Learning image blind denoisers without explicit noise modeling. Multimed Tools Appl 82, 27839–27859 (2023). https://doi.org/10.1007/s11042-023-14590-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14590-z

Keywords

Search

Navigation