Skip to main content
SpringerLink
Log in

Triple discriminators - equipped GAN for Denoising of Chinese calligraphic tablet images

  • 1221: Deep Learning for Image/Video Compression and Visual Quality Assessment
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Denoising of Chinese calligraphic tablet images is of great importance in regard to the study of both content and character shapes in these images. Formerly GAN (generative adversarial network) based image denoising methods model the noise in the generator and then perform denoising by CNN (convolutional neural networks) algorithms. These methods still leave room for improvement. In this paper, a triple discriminators equipped GAN for generative denoising is proposed, with the three channels of discriminators enhancing the denoising result by different means. Another noise modeling module based on CycleGAN is used to produce the paired input data. Quantitative index are obtained for these methods; the PSNR and SSIM of our method on publicly available data is 21.84 and 0.93 respectively, which is preferable to BM3D, DnCNN, FormResNet, CycleGAN and our previous method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Barham P, Chen J, Chen Z (n.d.) TensorFlow : a system for large-scale machine learning, OSDI’16: proceedings of the 12th USENIX conference on operating systems design and implementation, URL https://doi.org/10.5555/3026877.3026899

  2. Buades A, Coll B, Morel J-M (2011) URL. Non-Local Means Denoising, Computer Vision and Pattern Recognition 1:208–212. https://doi.org/10.5201/ipol.2011.bcm_nlm

    Article  Google Scholar 

  3. Chen Y, Lai YK, Liu YJ (2018) CartoonGAN: generative adversarial networks for photo Cartoonization, IEEE/CVF Conference on Computer Vision and Pattern Recognition. URL https://doi.org/10.1109/CVPR.2018.00986.

  4. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-D transform-domain collaborative filtering. IEEE Trans. Image Process. 16:2080–2095. https://doi.org/10.1109/TIP.2007.901238

    Article  MathSciNet  Google Scholar 

  5. El Helou M, Susstrunk S (2020) Blind Universal Bayesian Image Denoising with Gaussian Noise Level Learning. IEEE Trans Image Process 29:4885–4897. https://doi.org/10.1109/TIP.2020.2976814

    Article  MATH  Google Scholar 

  6. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15:3736–3745. https://ieeexplore.ieee.org/document/4011956. Accessed 13 Nov 2006

  7. Goodfellow I, NIPS (2016) Tutorial: generative adversarial networks, (2016). https://arxiv.org/abs/1701.00160. Accessed 31 Dec 2016

  8. Goodfellow IJ, Pouget-Abadie J, Mirza M (2014) Generative adversarial nets. Neural Inf Process Syst (NIPS 2014):2672–2680. https://doi.org/10.5555/2969033.2969125

  9. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition, Computer Science. URL https://www.computer.org/csdl/proceedings-article/cvpr/2016/8851a770/12OmNxvwoXv

  10. Hou B, Liu Q, Wang H, Wang Y (2020) From W-Net to CDGAN : Bi-temporal Change Detection via Deep Learning Techniques, IEEE Transact Geosci Remote Sens 1–12. URL https://ieeexplore.ieee.org/document/8891676

  11. Huang G, Weinberger KQ (n.d.) Densely Connected Convolutional Networks, IEEE Conference on Computer Vision and Pattern Recognition (CVPR),URL https://ieeexplore.ieee.org/document/8099726

  12. Huang ZK, Li ZH, Huang H, Li ZB, Hou LY (2016) Comparison of different image denoising algorithms for Chinese calligraphy images. Neurocomputing 188:102–112. https://doi.org/10.1016/j.neucom.2014.11.106

  13. Isola P, Efros AA, Ai B, Berkeley UC (2018) Image-to-image translation with conditional adversarial networks, IEEE conference on computer vision and pattern recognition. http://openaccess.thecvf.com/content_cvpr_2017/papers/Isola_Image-To-Image_Translation_With_CVPR_2017_paper.pdf. Accessed 9 Nov 2017

  14. Jain V, Seung HS (2009) Natural image denoising with convolutional networks, Conference on Advances in Neural Information Processing Systems Curran Associates Inc, 769–776. https://doi.org/10.5555/2981780.2981876

  15. Ji G, Wei Q, Wang KL (2020) In-air handwritten Chinese text recognition with temporal convolutional recurrent network. Pattern Recognit

  16. Jia X, Liu S, Feng X, Zhang L (2019) Focnet: A fractional optimal control network for image denoising. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://ieeexplore.ieee.org/document/8954104. Accessed 9 Jan 2020

  17. Jiao J, Tu WC, He S, Lau RWH (2017) FormResNet: Formatted residual learning for image restoration. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). https://ieeexplore.ieee.org/document/8014874. Accessed 24 Aug 2017

  18. D.P. Kingma, J.L. Ba, Adam (2014) A method for stochastic optimization, Computer Science 1–15. URL http://www.oalib.com/paper/4068193

  19. C. Luo, L. Jin, Z. Sun, I. Engineering, MORAN: a multi-object rectified attention network for scene text recognition, pattern Recognitiont. (2019). URL https://doi.org/10.1016/j.patcog.2019.01.020

  20. Maas AL, Ng A.Y (2013) Rectifier nonlinearities improve neural network acoustic models, international conference on machine learning 28. https://www.mendeley.com/catalogue/a4a3dd28-b56b-3e0c-ac53-2817625a2215/. Accessed 30 Sept 2013

  21. Mao X-J, Shen C, Yang Y-B (2016) Image Restoration Using Convolutional Auto-encoders with Symmetric Skip Connections, Conference on Neural Information Processing Systems. URL https://dblp.uni-trier.de/rec/journals/corr/MaoSY16a.html

  22. My VD,Manh ND,Phuong LT, Sang-Woong L (2021) HI-GAN: A hierarchical generative adversarial network for blind denoising of real photographs[J]. Inf Sci, 570. Access date: [2021-03-06].

  23. Wang Z, Bovik AC, Sheikh HR (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13:600–612. ieeexplore.ieee.org/document/1284395. Accessed 13 Apr 2004

  24. Wu Y, Shutao L (2021) A novel fusion paradigm for multi-channel image denoising[J]. Inf Fusion 77:62–69

  25. Xia Y, He D, Qin T (2016) Dual learning for machine translation, neural information processing systems. http://papers.nips.cc/paper/6469-dual-learning-for-machine-translation. Accessed 31 Dec 2016

  26. Xie J, Xu L, Chen E (2012) Image denoising and inpainting with deep neural networks. Advances in neural information processing systems. https://doi.org/10.5555/2999134.2999173

  27. Zach C, Eth Z, Klopschitz M, Pollefeys M, Eth Z (2010) Disambiguating visual relations using loop constraints. IEEE computer society conference on computer vision and pattern recognition. URL https://ieeexplore.ieee.org/document/5539801

  28. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising. IEEE Trans Image Process 26(2017):3142–3155. https://doi.org/10.1109/TIP.2017.2662206, 3142, 3155

  29. Zhang K, Zuo W, Zhang L (2018) FFDNet: Toward a fast and flexible solution for CNN-Based image denoising. IEEE Trans Image Process 27:4608–4622. https://doi.org/10.1109/TIP.2018.2839891

  30. Zhang Y, Tian Y, Kong Y (2018) Residual dense network for image super-resolution. In: IEEE conference on computer vision and pattern recognition. http://openaccess.thecvf.com/content_cvpr_2018/html/1329.html. Accessed 27 Dec 2018

  31. Zhang J, Guo M, Fan J (2020) A novel generative adversarial net for calligraphic tablet images denoising. Multimed Tools Appl 79:119–140. https://doi.org/10.1007/s11042-019-08052-8

    Article  Google Scholar 

  32. J.Y. Zhu, T. Park, P. Isola, A.A. Efros (2017) Unpaired image-to-image translation using cycle-consistent adversarial networks, IEEE Int Conf Computer Vis https://doi.org/10.1109/ICCV.2017.244.

Download references

Acknowledgements

This work is supported under the support of funds of Shaanxi Department of science and technology  2022QFY01-17, 2022JM-326. We thank Professor Songhua Xu for his kind discussion of the method in the paper.

Author information

Authors and Affiliations

  1. Institute of Computer Science and Engineering, Xi’an University of Technology, Xi’an, China

    Jiulong Zhang, Jiaxi Shi & Mengyang Li

  2. Shaanxi Key Laboratory for Network Computing and Security Technology, Xi’an, China

    Jiulong Zhang

  3. National Key Laboratory of Fundamental Science on Synthetic Vision, Sichuan University, Chengdu, China

    Mingtao Guo

  4. Nanjing University of Information Science & Technology, Nanjing, China

    Zhigeng Pan

Authors
  1. Jiulong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiaxi Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengyang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mingtao Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhigeng Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jiulong Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, J., Shi, J., Li, M. et al. Triple discriminators - equipped GAN for Denoising of Chinese calligraphic tablet images. Multimed Tools Appl 81, 42691–42711 (2022). https://doi.org/10.1007/s11042-022-13478-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13478-8

Keywords

Search

Navigation