Skip to main content
SpringerLink
Log in

Total generalized variational-liked network for image denoising

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Deep convolutional neural networks (DCNN) have been widely used in the field of image denoising because of their fast inference and good performance. However, the design of networks for the DCNN is mostly empirical, and the interpretation and robustness of them remains a major challenge. Inspired by the total generalized variation method, this paper proposes a novel adaptive denoising network. It mainly improves the TGV algorithm in terms of two points. Firstly, the first and second order derivation term are replaced by the learnable operators. Secondly, the regularization terms are learned from the training data by using convolution networks other than the fixed ones. The network design derived from the process for tackling the denoising problem based on the primal-dual hybrid gradient optimization algorithm, is called TGVLNet- Total Generalized Variational-Liked Network, which allows for the image prior and the linear operators to be tuned differently in each iteration, and enhances the flexibility and generalization ability of the network. The experiment results of Gaussian noise removal and signal dependent noise removal manifest that the proposed network has superior performance and generalization. Compared to most of the blind denoising methods with additive white Gaussian noise, the proposed TGVLNet performs better in unseen noise level. It is noting that we train the model only on the synthetic image of the signal dependent noise removal, and use the model to remove the noise of some images on two real datasets, i.e NC12 and Nam, the denoising results also presents much better visual quality and performance, which further verify the generalization and robustness of our method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Arbelaez P, Fowlkes C, Martin D (2007) The Berkeley segmentation dataset and benchmark. Computer Science Department, Berkeley University [Online]. Available: http://www.eecs.berkeley.edu/Research/Projects/CS/vision/bsds

  2. Bredies K, Dong Y, Hintermüller M. (2013) Spatially dependent regularization parameter selection in total generalized variation models for image restoration. Int J Comput Math 90(1):109–123

    Article  MathSciNet  MATH  Google Scholar 

  3. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: IEEE Computer society conference on computer vision and pattern recognition (CVPR’05), vol 2, pp 60–65, IEEE

  4. Burger HC, Schuler CJ, Harmeling S (2012) Image denoising: can plain neural networks compete with bm3d?. In: IEEE Conference on computer vision and pattern recognition, pp 2392–2399, IEEE

  5. Chen Y, Pock T (2016) Trainable nonlinear reaction diffusion: a flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell 39(6):1256–1272

    Article  Google Scholar 

  6. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Color image denoising via sparse 3d collaborative filtering with grouping constraint in luminance-chrominance space. In: IEEE International conference on image processing, vol 1, pp I–313, IEEE

  7. Dabov K, Foi A, Katkovnik V, Egiazarian K (2007) Image denoising by sparse 3-d transform-domain collaborative filtering. IEEE Trans Image Process 16(8):2080–2095

    Article  MathSciNet  Google Scholar 

  8. Dong C, Loy CC, He K, Tang X (2015) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  9. Du H (2019) Image denoising algorithm based on nonlocal regularization sparse representation IEEE Sensors Journal

  10. Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15(12):3736–3745

    Article  MathSciNet  Google Scholar 

  11. Getreuer P (2012) Rudin-osher-fatemi total variation denoising using split bregman. Image Processing On Line 2:74–95

    Article  Google Scholar 

  12. Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869

  13. Guo S, Yan Z, Zhang K, Zuo W, Zhang L (2019) Toward convolutional blind denoising of real photographs. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1712–1722

  14. Jain V, Seung S (2009) Natural image denoising with convolutional networks. In: Advances in neural information processing systems, pp 769–776

  15. Lai R, Mo Y, Liu Z, Guan J (2019) Local and nonlocal steering kernel weighted total variation model for image denoising. Symmetry 11(3):329

    Article  Google Scholar 

  16. Lebrun M, Colom M, Morel JM (2015) The noise clinic: a blind image denoising algorithm. Image Processing On Line 5:1–54

    Article  MATH  Google Scholar 

  17. Lefkimmiatis S (2018) Universal denoising networks: a novel cnn architecture for image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3204–3213

  18. Liu X, Tanaka M, Okutomi M (2014) Practical signal-dependent noise parameter estimation from a single noisy image. IEEE Trans Image Process 23(10):4361–4371

    Article  MathSciNet  MATH  Google Scholar 

  19. Loui A, Luo J, Chang SF, Ellis D, Jiang W, Kennedy L, Lee K, Yanagawa A (2007) Kodak’s consumer video benchmark data set: concept definition and annotation. In: Proceedings of the international workshop on Workshop on multimedia information retrieval, pp 245–254, ACM

  20. Ma K, Duanmu Z, Wu Q, Wang Z, Yong H, Li H, Zhang L (2016) Waterloo exploration database: new challenges for image quality assessment models. IEEE Trans Image Process 26(2):1004–1016

    Article  MathSciNet  MATH  Google Scholar 

  21. Mairal J, Elad M, Sapiro G (2007) Sparse representation for color image restoration. IEEE Trans Image Process 17(1):53–69

    Article  MathSciNet  Google Scholar 

  22. Mao X, Shen C, Yang YB (2016) Image restoration using very deep convolutional encoder-decoder networks with symmetric skip connections. In: Advances in neural information processing systems, pp 2802–2810

  23. Martin D, Fowlkes C, Tal D, Malik J, et al. (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE International conference on computer vision. Iccv vancouver

  24. Nam S, Hwang Y, Matsushita Y, Joo Kim S (2016) A holistic approach to cross-channel image noise modeling and its application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1683–1691

  25. Quan Y, Chen Y, Shao Y, Teng H, Xu Y, Ji H (2021) Image denoising using complex-valued deep cnn. Pattern Recogn 111:107–639. https://doi.org/10.1016/j.patcog.2020.107639

    Article  Google Scholar 

  26. Shahdoosti HR, Rahemi Z (2019) Edge-preserving image denoising using a deep convolutional neural network. Signal Process 159:20–32

    Article  Google Scholar 

  27. Shen Y, Liu Q, Lou S, Hou YL (2017) Wavelet-based total variation and nonlocal similarity model for image denoising. IEEE Signal Process Lett 24(6):877–881

    Article  Google Scholar 

  28. Singh G, Mittal A, Aggarwal N (2020) Resdnn: deep residual learning for natural image denoising. IET Image Process 14(11):2425–2434. https://doi.org/10.1049/iet-ipr.2019.0623

    Article  Google Scholar 

  29. Tai Y, Yang J, Liu X, Xu C (2017) Memnet: a persistent memory network for image restoration. In: Proceedings of the IEEE international conference on computer vision, pp 4539–4547

  30. Tian C, Xu Y, Li Z, Zuo W, Fei L, Liu H (2020) Attention-guided cnn for image denoising. Neural Netw 124:117–129. https://doi.org/10.1016/j.neunet.2019.12.024

    Article  Google Scholar 

  31. Tian C, Xu Y, Zuo W (2020) Image denoising using deep cnn with batch renormalization. Neural Netw 121:461–473. https://doi.org/10.1016/j.neunet.2019.08.022

    Article  Google Scholar 

  32. Tian C, Xu Y, Zuo W, Du B, Lin CW, Zhang D (2021) Designing and training of a dual cnn for image denoising. Knowl-Based Syst 226:106–949. https://doi.org/10.1016/j.knosys.2021.106949

    Article  Google Scholar 

  33. Wali S, Zhang H, Chang H, Wu C (2019) A new adaptive boosting total generalized variation (tgv) technique for image denoising and inpainting. J Vis Commun Image Represent 59:39–51

    Article  Google Scholar 

  34. Xie J, Xu L, Chen E (2012) Image denoising and inpainting with deep neural networks. In: Advances in neural information processing systems, pp 341–349

  35. xu Y, Tian C, Fei L, Wang J, Jie W, Luo N (2019) Enhanced cnn for image denoising CAAI Transactions on Intelligence Technology, vol 4 https://doi.org/10.1049/trit.2018.1054

  36. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  MathSciNet  MATH  Google Scholar 

  37. Zhang K, Zuo W, Zhang L (2018) Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Trans Image Process 27(9):4608–4622

    Article  MathSciNet  Google Scholar 

  38. Zhang L, Wu X, Buades A, Li X (2011) Color demosaicking by local directional interpolation and nonlocal adaptive thresholding. J Electron Imaging 20(2):023–016

    Google Scholar 

  39. Zhang X, Liu J, Lu Y, Dong B (2019) Dynamically unfolding recurrent restorer: a moving endpoint control method for image restoration. In: 7Th international conference on learning representations, ICLR

  40. Zhao H, Shao W, Bao B, Li H (2019) A simple and robust deep convolutional approach to blind image denoising. In: Proceedings of the IEEE international conference on computer vision workshops, pp 0–0

  41. Zhu J, Li K, Hao B (2019) Restoration of remote sensing images based on nonconvex constrained high-order total variation regularization. J Appl Remote Sens 13(2):022–006

    Article  Google Scholar 

  42. Zuo C, Jovanov L, Luong HQ, Goossens B, Philips W, Liu Y, Zhang M (2015) Rotation invariant similarity measure for non-local self-similarity based image denoising. In: IEEE International conference on image processing (ICIP), pp 1618–1622, IEEE

Download references

Author information

Authors and Affiliations

  1. School of Information & Science and Engineering, Yanshan University, Qinhuangdao, China

    Zhang Xiaohua, Lian Qiusheng & Zhang Dan

  2. School of Mathematics and Information Science & Technology, Hebei Normal University of Science & Technology, Qinhuangdao, China

    Zhang Xiaohua

  3. Hebei Key Laboratory of Information Transmission and Signal Processing, Qinhuangdao, China

    Lian Qiusheng & Zhang Dan

Authors
  1. Zhang Xiaohua
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lian Qiusheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhang Dan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lian Qiusheng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Natural Science Foundation of China under Grant No.61471313 and Hebei Natural Science Foundation F2019203318.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiaohua, Z., Qiusheng, L. & Dan, Z. Total generalized variational-liked network for image denoising. Appl Intell 53, 9650–9667 (2023). https://doi.org/10.1007/s10489-022-03717-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03717-8

Keywords

Search

Navigation