Skip to main content
SpringerLink
Log in

Perceptual adversarial non-residual learning for blind image denoising

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Image denoising is tricky work required in various image processing and computer vision challenges. This paper proposes and implements a perceptual adversarial non-residual blind denoising training architecture based on non-residual adversarial learning for spatially variant blind image denoising challenges. The image denoising techniques based on the residual learning encounter an image artifacts problem on higher noise levels, fail to construct target-oriented images, and hence are limited to spatially invariant single image denoising tasks. Additionally, denoising techniques based on convolutional neural networks encounter a blurriness problem, resulting in blurry output texture of images. To solve the blind image denoising tasks and blurriness problem, we first conduct a theoretical investigation into the underlying cause. We then proposed the Non-Residual Blind Image Denoising Network, leveraging the generative adversarial network-based adversarial and non-residual learning procedures for blind image denoising applications. The proposed framework removes multiple spatially variant noises from given noisy images and constructs visually pleasant images with a single model. This model resolves the image artifacts and blurriness issue by motivating the denoising network to discover the sharp, perceptually strong, target-oriented noise-free distribution. Extensive experimental results show that the proposed method can efficiently resolve the image artifacts and blurriness issue and accomplish significant denoising proficiency than the state-of-the-art denoising techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availability

Enquiries about data availability should be directed to the authors.

References

  • Arjovsky M, Chintala S, Bottou L (2017) Wasserstein generative adversarial networks. In: Proceedings of international conference on machine learning, pp 214–223

  • Barbu A (2009) Training an active random field for real-time image denoising. IEEE Trans Image Process 18:2451–2462

    Article  MathSciNet  Google Scholar 

  • Buades A, Coll B, Morel J-M (2005) A non-local algorithm for image denoising. In: Proceedings of 2005 IEEE computer society conference on computer vision and pattern recognition (CVPR'05), pp 60–65

  • Chakrabarty N (2020) Brain MRI images for brain tumor detection | Kaggle n.d. https://www.kaggle.com/navoneel/brain-mri-images-for-brain-tumor-detection. Accessed 25 Aug 2020

  • Chen Y, Yu W, Pock T (2015) On learning optimized reaction diffusion processes for effective image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5261–5269

  • Chen X, Duan Y, Houthooft R, Schulman J, Sutskever I, Abbeel P (2016) Infogan: interpretable representation learning by information maximizing generative adversarial nets. In: Proceedings of advances in neural information processing systems, pp 2172–2180

  • Chen Y, Lai Y-K, Liu Y-J (2017) Transforming photos to comics using convolutional neural networks. In: Proceedings of 2017 IEEE international conference on image processing (ICIP), pp 2010–2014

  • Chen X, Xu C, Yang X, Song L, Tao D (2018a) Gated-gan: adversarial gated networks for multi-collection style transfer. IEEE Trans Image Process 28:546–560

    Article  MathSciNet  Google Scholar 

  • Chen J, Chen J, Chao H, Yang M (2018b) Image blind denoising with generative adversarial network based noise modeling. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3155–3164

  • Chen S, Shi D, Sadiq M, Cheng X (2020) Image denoising with generative adversarial networks and its application to cell image enhancement. IEEE Access 8:82819–82831

    Article  Google Scholar 

  • Cheng Z, Yang Q, Sheng B (2015) Deep colorization. In: Proceedings of the IEEE international conference on computer vision, pp 415–423

  • Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16:2080–2095

    Article  MathSciNet  Google Scholar 

  • Dong W, Zhang L, Shi G, Li X (2012) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22:1620–1630

    Article  MathSciNet  Google Scholar 

  • Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38:295–307

    Article  Google Scholar 

  • Dong W, Wang P, Yin W, Shi G, Wu F, Lu X (2018) Denoising prior driven deep neural network for image restoration. IEEE Trans Pattern Anal Mach Intell 41:2305–2318

    Article  Google Scholar 

  • Du B, Wei Q, Liu R (2019) An improved quantum-behaved particle swarm optimization for endmember extraction. IEEE Trans Geosci Remote Sens 57:6003–6017

    Article  Google Scholar 

  • Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15:3736–3745

    Article  MathSciNet  Google Scholar 

  • Fan L, Zhang F, Fan H, Zhang C (2019) Brief review of image denoising techniques. Vis Comput Ind Biomed Art 2:1–12

    Article  Google Scholar 

  • Ge H, Yao Y, Chen Z, Sun L (2018) Unsupervised transformation network based on GANs for target-domain oriented image translation. IEEE Access 6:61342–61350

    Article  Google Scholar 

  • Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of advances in neural information processing systems, pp 2672–2680

  • Goyal B, Dogra A, Agrawal S, Sohi B, Sharma A (2020) Image denoising review: from classical to state-of-the-art approaches. Inf Fusion 55:220–244

    Article  Google Scholar 

  • Gu S, Timofte R (2019) A brief review of image denoising algorithms and beyond. In: Escalera S, Ayache S, Wan J et al (eds) Inpainting and denoising challenges. Springer, Cham, pp 1–21

    Google Scholar 

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  • Heusel M, Ramsauer H, Unterthiner T, Nessler B, Hochreiter S (2017) Gans trained by a two time-scale update rule converge to a local nash equilibrium. In: Proceedings of advances in neural information processing systems, pp 6626–6637

  • Isola P, Zhu J-Y, Zhou T, Efros AA (2017) Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1125–1134

  • Jain V, Seung S (2008) Natural image denoising with convolutional networks. Adv Neural Inf Process Syst 21:769–776

    Google Scholar 

  • Jancsary J, Nowozin S, Sharp T, Rother C (2012) Regression tree fields: an efficient, non-parametric approach to image labeling problems. In: Proceedings of 2012 IEEE conference on computer vision and pattern recognition, pp 2376–2383

  • Jie W, Yong X, Hong L (2020) Incomplete multiview spectral clustering with adaptive graph learning. IEEE Trans Cybern 50:1418–1429

    Article  Google Scholar 

  • Jifara W, Jiang F, Rho S, Cheng M, Liu S (2019) Medical image denoising using convolutional neural network: a residual learning approach. J Supercomput 75:704–718

    Article  Google Scholar 

  • Johnson J, Alahi A, Fei-Fei L (2016) Perceptual losses for real-time style transfer and super-resolution. In: Proceedings of European conference on computer vision, pp 694–711

  • Khan A, Jin W, Ahmad M, Naqvi RA, Wang D (2020) An input-perceptual reconstruction adversarial network for paired image-to-image conversion. Sensors 20:4161

    Article  Google Scholar 

  • Khan A, Jin W, Haider A, Rahman M, Wang D (2021) Adversarial Gaussian denoiser for multiple-level image denoising. Sensors 21:2998

    Article  Google Scholar 

  • Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  • Krull A, Buchholz T-O, Jug F (2019) Noise2void-learning denoising from single noisy images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 2129–2137

  • Kupyn O, Budzan V, Mykhailych M, Mishkin D, Matas J (2018) Deblurgan: blind motion deblurring using conditional adversarial networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8183–8192

  • Lan X, Roth S, Huttenlocher D, Black MJ (2006) Efficient belief propagation with learned higher-order markov random fields. In: Proceedings of European conference on computer vision, pp 269–282

  • Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z (2017) Photo-realistic single image super-resolution using a generative adversarial network. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4681–4690

  • Lehtinen J, Munkberg J, Hasselgren J, Laine S, Karras T, Aittala M, Aila T (2018) Noise2noise: learning image restoration without clean data. arXiv preprint arXiv:1803.04189

  • Li C, Wand M (2016) Precomputed real-time texture synthesis with Markovian generative adversarial networks. In: Proceedings of European conference on computer vision, pp 702–716

  • Li J, Liang X, Wei Y, Xu T, Feng J, Yan S (2017) Perceptual generative adversarial networks for small object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1222–1230

  • Liu D, Wen B, Liu X, Wang Z, Huang TS (2017) When image denoising meets high-level vision tasks: a deep learning approach. arXiv preprint arXiv:1706.04284

  • Liu P, El Basha MD, Li Y, Xiao Y, Sanelli PC, Fang R (2019) Deep evolutionary networks with expedited genetic algorithms for medical image denoising. Med Image Anal 54:306–315

    Article  Google Scholar 

  • Liu D, Wen B, Jiao J, Liu X, Wang Z, Huang TS (2020a) Connecting image denoising and high-level vision tasks via deep learning. IEEE Trans Image Process 29:3695–3706

    Article  Google Scholar 

  • Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020b) Deep learning for generic object detection: a survey. Int J Comput Vis 128:261–318

    Article  Google Scholar 

  • Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440

  • Lotter W, Kreiman G, Cox D (2015) Unsupervised learning of visual structure using predictive generative networks. arXiv preprint arXiv:1511.06380

  • Mairal J, Elad M, Sapiro G (2007) Sparse representation for color image restoration. IEEE Trans Image Process 17:53–69

    Article  MathSciNet  Google Scholar 

  • Mairal J, Bach F, Ponce J, Sapiro G, Zisserman A (2009) Non-local sparse models for image restoration. In: Proceedings of 2009 IEEE 12th international conference on computer vision, pp 2272–2279

  • Mao X-J, Shen C, Yang Y-B (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. arXiv preprint arXiv:1603.09056

  • Mao X, Li Q, Xie H, Lau RY, Wang Z, Paul Smolley S (2017) Least squares generative adversarial networks. In: Proceedings of the IEEE international conference on computer vision, pp 2794–2802

  • Mirza M, Osindero S (2014) Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784

  • Regmi K, Borji A (2018) Cross-view image synthesis using conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3501–3510

  • Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M (2015) Imagenet large scale visual recognition challenge. Int J Comput Vis 115:211–252

    Article  MathSciNet  Google Scholar 

  • Schmidt U, Roth S (2014) Shrinkage fields for effective image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2774–2781

  • Sharif S, Naqvi RA, Biswas M (2020) Learning medical image denoising with deep dynamic residual attention network. Mathematics 8:2192

    Article  Google Scholar 

  • Sheikh HR, Bovik AC (2006) Image information and visual quality. IEEE Trans Image Process 15:430–444

    Article  Google Scholar 

  • Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu K-I, Matsui M, Fujita H, Kodera Y, Doi K (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am J Roentgenol 174:71–74

    Article  Google Scholar 

  • Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556

  • Southwest Jiaotong University (ed) (2012) The high speed railway power supply safety inspection and monitoring system (6C) general technical specification. Transport power supply Department of MOR, C.A.o.R.S., Southwest Jiaotong University

  • Sun J, Tappen MF (2011) Learning non-local range Markov random field for image restoration. In: Proceedings of CVPR 2011, pp 2745–2752

  • Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1891–1898

  • Tang P, Jin W, Liu J (2016) Railway inspection oriented foreground objects detection and occlusion reasoning for locomotive-mounted camera video. In: Proceedings of 2016 35th Chinese control conference (CCC), pp 10144–10149

  • Timofte R, Agustsson E, Van Gool L, Yang M-H, Zhang L (2017) Ntire 2017 challenge on single image super-resolution: Methods and results. In: Proceedings of the IEEE conference on computer vision and pattern recognition workshops, pp 114–125

  • Uddin AS, Chung T, Bae S-H (2019) A perceptually inspired new blind image denoising method using L1 and perceptual loss. IEEE Access 7:90538–90549

    Article  Google Scholar 

  • Ulyanov D, Vedaldi A, Lempitsky V (2016) Instance normalization: the missing ingredient for fast stylization. arXiv preprint arXiv:1607.08022

  • Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Process Lett 9:81–84

    Article  Google Scholar 

  • Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612

    Article  Google Scholar 

  • Wang C, Xu C, Wang C, Tao D (2018a) Perceptual adversarial networks for image-to-image transformation. IEEE Trans Image Process 27:4066–4079. https://doi.org/10.1109/TIP.2018.2836316

    Article  MathSciNet  MATH  Google Scholar 

  • Wang T-C, Liu M-Y, Zhu J-Y, Tao A, Kautz J, Catanzaro B (2018b) High-resolution image synthesis and semantic manipulation with conditional gans. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 8798–8807

  • Wang D, Jin W, Wu Y, Khan A (2021) Improving global adversarial robustness generalization with adversarially trained GAN. arXiv preprint arXiv:2103.04513

  • Wen J, Zhang Z, Zhang Z, Fei L, Wang M (2020) Generalized incomplete multiview clustering with flexible locality structure diffusion. IEEE Trans Cybern 51:101–114

    Article  Google Scholar 

  • Xie J, Xu L, Chen E (2012) Image denoising and inpainting with deep neural networks. Adv Neural Inf Process Syst 25:341–349

    Google Scholar 

  • Xu J, Zhang L, Zhang D (2018) External prior guided internal prior learning for real-world noisy image denoising. IEEE Trans Image Process 27:2996–3010

    Article  MathSciNet  Google Scholar 

  • Yu F, Koltun V (2015) Multi-scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122

  • Zha Z, Yuan X, Wen B, Zhou J, Zhang J, Zhu C (2019) From rank estimation to rank approximation: rank residual constraint for image restoration. IEEE Trans Image Process 29:3254–3269

    Article  MathSciNet  Google Scholar 

  • Zhang R, Isola P, Efros AA (2016) Colorful image colorization. In: Proceedings of European conference on computer vision, pp 649–666

  • Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017a) Beyond a gaussian denoiser: residual learning of deep cnn for image denoising. IEEE Trans Image Process 26:3142–3155

    Article  MathSciNet  Google Scholar 

  • Zhang K, Zuo W, Gu S, Zhang L (2017b) Learning deep CNN denoiser prior for image restoration. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3929–3938

  • Zhang K, Zuo W, Zhang L (2018) FFDNet: toward a fast and flexible solution for CNN-based image denoising. IEEE Trans Image Process 27:4608–4622

    Article  MathSciNet  Google Scholar 

Download references

Funding

The authors have not disclosed any funding.

Author information

Authors and Affiliations

  1. School of Electrical Engineering, Southwest Jiaotong University, Chengdu, 611756, China

    Aamir Khan & Weidong Jin

  2. China-ASEAN International Joint Laboratory of Integrated Transport, Nanning University, Nanning, 530000, China

    Weidong Jin

  3. Department of Unmanned Vehicle Engineering, Sejong University, Seoul, 05006, Korea

    Rizwan Ali Naqvi

Authors
  1. Aamir Khan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weidong Jin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rizwan Ali Naqvi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weidong Jin.

Ethics declarations

Conflict of interest

The authors have not disclosed any conflict of interest.

Additional information

Communicated by Irfan Uddin.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khan, A., Jin, W. & Naqvi, R.A. Perceptual adversarial non-residual learning for blind image denoising. Soft Comput 26, 7933–7957 (2022). https://doi.org/10.1007/s00500-022-06853-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-022-06853-y

Keywords

Search

Navigation