Skip to main content
SpringerLink
Log in

Pyramidal convolution attention generative adversarial network with data augmentation for image denoising

  • Data analytics and machine learning
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Generative adversarial networks (GANs) have shown remarkable effects for various computer vision tasks. Standard convolution plays an important role in the GAN-based model. However, the single type of kernel with a single spatial size limits the learning ability of the model and does not explicitly consider the dependencies among channels. To overcome these issues, this paper proposes a pyramidal convolution attention GAN for image denoising, a model that uses a residual structure with a pyramidal convolution attention block (PyCA) instead of the stacked standard convolution as a generator within the GAN setting. The proposed PyCA considers the channel-wise dependencies while extracting multi-scale features. Besides, we also design a data augmentation method for image denoising. The experimental results show that our model achieves better denoising performance than other competing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Aamir M, Nawi NM, Mahdin HB, Naseem R, Zulqarnain M (2020) Auto-encoder variants for solving handwritten digits classification problem. Int J Fuzzy Log Intell Syst 20(1):8–16

    Article  Google Scholar 

  • Arbeláez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  • Brock A, Donahue J, Simonyan K (2019) Large scale gan training for high fidelity natural image synthesis. In: 2019 international conference on learning representations (ICLR)

  • Chauhan N, Choi BJ (2018) Performance analysis of denoising algorithms for human brain image. Int J Fuzzy Log Intell Syst 18(3):175–181

    Article  Google Scholar 

  • Chauhan N, Choi BJ (2019) Denoising approaches using fuzzy logic and convolutional autoencoders for human brain MRI image. Int J Fuzzy Log Intell Syst 19(3):135–139

    Article  Google Scholar 

  • Chen Y, Fan H, Xu B, Yan Z, Kalantidis Y, Rohrbach M, Yan S, Feng J (2019) Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution. In: 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, pp 3434–3443

  • Chen Y, Dai X, Liu M, Chen D, Yuan L, Liu Z (2020) Dynamic convolution: Attention over convolution kernels. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, pp 11027–11036

  • Choi SH, Jung SH (2019) Similarity analysis of actual fake fingerprints and generated fake fingerprints by dcgan. Int J Fuzzy Log Intell Syst 19(1):40–47

    Article  Google Scholar 

  • Dabov K, Foi A, Katkovnik V, Egiazarian K (2007a) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    Article  MathSciNet  Google Scholar 

  • Dabov K, Foi A, Katkovnik V, Egiazarian K (2007b) Color image denoising via sparse 3D collaborative filtering with grouping constraint in luminance-chrominance space. In: Proceedings of the IEEE international conference on image processing (ICIP). IEEE, pp 313–316

  • DeVries T, Taylor GW (2017) Improved regularization of convolutional neural networks with cutout. ArXiv preprint arXiv:1708.04552

  • Dong W, Zhang L, Shi G, Li X (2013) Nonlocally centralized sparse representation for image restoration. IEEE Trans Image Process 22(4):1620–1630

    Article  MathSciNet  Google Scholar 

  • Duta IC, Liu L, Zhu F, Shao L (2020) Pyramidal convolution: rethinking convolutional neural networks for visual recognition. ArXiv preprint arXiv:2006.11538

  • Gastaldi X (2017) Shake-shake regularization. ArXiv preprint arXiv:1705.07485

  • Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the 2014 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 580–587

  • Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville A, Bengio Y (2014) Generative adversarial nets. In: Proceedings of the 28th annual conference on neural information proceeding system (NeurIPS), pp 2672–2680

  • Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 2862–2869

  • He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 770–778

  • Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Processing of the 2018 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, pp 7132–7141

  • Kligvasser I, Shaham TR, Michaeli T (2018) xUnit: learning a spatial activation function for efficient image restoration. In: 2018 IEEE/CVF conference on computer vision and pattern recognition (CPVR). IEEE, pp 2433–2442

  • Lu Z, Deb K, Naresh Boddeti V (2020) MUXConv: information multiplexing in convolutional neural networks. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, pp 12041–12050

  • Liu JJ, Hou Q, Cheng MM, Wang C, Feng J (2020) Improving convolutional networks with self-calibrated convolutions. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, pp 10093–10102

  • Tang H, Liu H, Sebe N (2020) Unified generative adversarial networks for controllable image-to-image translation. IEEE Trans Image Process 29:8916–8929

    Article  Google Scholar 

  • Tariqul IM, Mahbubur RSM, Omair AM, Swamy MNS (2018) Mixed gaussian-impulse noise reduction from images using convolutional neural network. Signal Process Image Commun 68:26–41

    Article  Google Scholar 

  • Radford A, Metz L, Chintala S (2016) Unsupervised representation learning with deep convolutional generative adversarial networks. In: 2016 international conference on learning representations (ICLR)

  • Redmon JC, Pascal VOC dataset mirror (VOC 2007). https://pjreddie.com/projects/pascal-voc-dataset-mirror/

  • Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: 2015 international conference on learning representations (ICLR)

  • Tan M, Le Quoc V (2019) MixConv: mixed depthwise convolutional kernels. ArXiv preprint arXiv:1907.09595

  • Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  • Wei K, Fu Y, Yang J, Huang H (2020) A physics-based noise formation model for extreme low-light raw denoising. In: Proceedings of the 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, pp 2755–2764

  • Xu J, Zhang L, Zuo W, Zhang D, Feng X (2015) Patch group based nonlocal self-similarity prior learning for image denoising. In: 2015 IEEE international conference on computer vision (ICCV). IEEE, pp 244–252

  • Xu J, Zhang L, Zhang D, Feng X (2017) Multi-channel weighted nuclear norm minimization for real color image denoising. In: 2017 IEEE international conference on computer vision (ICCV). IEEE, pp 1105–1113

  • Yang Q et al (2018) Low-dose CT image denoising using a generative adversarial network with wasserstein distance and perceptual loss. IEEE Trans Med Imaging 37(6):1348–1357

    Article  Google Scholar 

  • Yoo J, Ahn N, Sohn K (2020) Rethinking data augmentation for image super-resolution: a comprehensive analysis and a new strategy. In: 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR), IEEE, pp 8372–8381

  • You C et al (2018) Structurally-sensitive multi-scale deep neural network for low-dose CT denoising. IEEE Access 6:41839–41855

    Article  Google Scholar 

  • Yun S, Han D, Chun S, Oh SJ, Yoo Y, Choe J (2019) CutMix: regularization strategy to train strong classifiers with localizable features. In: 2019 IEEE/CVF international conference on computer vision (ICCV). IEEE, pp 6022–6031

  • Zhang H, Cisse M, Dauphin YN, Lopez-Paz D (2017a) Mixup: beyond empirical risk minimization. ArXiv preprint arXiv:1710.09412

  • Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017b) Beyond a gaussian denoiser: residual learning of deep CNN for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MathSciNet  Google Scholar 

  • Zhang Y, Li K, Li K, Wang L, Zhong B, Fu Y (2018) Image super-resolution using very deep residual channel attention networks. Process Eur Conf Comput vis 11211:294–310

    Google Scholar 

  • Zeyde R, Elad M, Protter M (2010) On single image scale-up using sparse-representations. In: Proceeding of the 7th international conference on curves and surfaces. Springer, Berlin, pp 711–730

  • Zoran D, Weiss Y (2011) From learning models of natural image patches to whole image restoration. In: 2011 international conference on computer vision (ICCV). IEEE, pp 479–486

Download references

Acknowledgements

We appreciated the help of the Henan Intelligent Traffic Safety Engineering Technology Research Center and Henan Multimodal Data Intelligent Traffic Safety Engineering Technology Research Center.

Author information

Authors and Affiliations

  1. School of Software, Pingdingshan University, Pingdingshan, 467000, China

    Qiongshuai Lyu & Dongliang Xia

  2. School of Computer Science, Shaanxi Normal University, Xi’an, 710119, China

    Qiongshuai Lyu, Yaling Liu, Xiaojing Yang & Rui Li

Authors
  1. Qiongshuai Lyu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongliang Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yaling Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaojing Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qiongshuai Lyu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lyu, Q., Xia, D., Liu, Y. et al. Pyramidal convolution attention generative adversarial network with data augmentation for image denoising. Soft Comput 25, 9273–9284 (2021). https://doi.org/10.1007/s00500-021-05870-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-021-05870-7

Keywords

Search

Navigation