Skip to main content
SpringerLink
Log in

A bilevel optimization problem with deep learning based on fractional total variation for image denoising

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this work, we introduce a bilevel problem based on fractional-order total variation for image denoising. A deep learning architecture is provided in the upper model to strengthen the regularization term in the lower one. Numerical results show the efficacy of the proposed approach, compared to various competitive models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Algorithm 2
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Hamalainen K, Kallonen A, Kolehmainen V, Lassas M, Niinimaki K, Siltanen S (2013) Sparse tomography. SIAM. J Sci Comput 35(3):644–665

    MathSciNet  Google Scholar 

  2. Girard DA (1987) Optimal regularized reconstruction in computerized tomography. SIAM J Sci Stat Comput 8(6):934–950

    Article  MathSciNet  Google Scholar 

  3. Hsieh J, Nett B, Yu Z, Sauer K, Thibault J-B, Bouman CA (2013) Recent advances in ct image reconstruction. Curr Radiol Rep 1(1):39–51

    Article  Google Scholar 

  4. Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Phys D Nonlinear Phenom 60(1–4):259–268

    Article  ADS  MathSciNet  Google Scholar 

  5. Taheri S, Gasparovic C, Shah NJ, Rosenberg GA (2011) Quantitative measurement of blood-brain barrier permeability in human using dynamic contrast-enhanced mri with fast t1 mapping. Magn Reson Med 65(4):1036–1042

    Article  PubMed  Google Scholar 

  6. Zhang K, Zuo W, Gu S, Zhang L (2017) Learning deep cnn denoiser prior for image restoration. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, p 3929–3938

  7. Lucas A, Iliadis M, Molina R, Katsaggelos AK (2018) Using deep neural networks for inverse problems in imaging: beyond analytical methods. IEEE Signal Proc Mag 35(1):20–36

    Article  ADS  Google Scholar 

  8. McCann MT, Jin KH, Unser M (2017) Convolutional neural networks for inverse problems in imaging: A review. IEEE Signal Proc Mag 34(6):85–95

    Article  ADS  Google Scholar 

  9. Chung J, Español MI (2017) Learning regularization parameters for general-form tikhonov. Inverse Problems 33(7):074004

    Article  ADS  MathSciNet  Google Scholar 

  10. Hansen PC (1989) Regularization, gsvd and truncated gsvd. BIT Numer Math 29:491–504

    Article  MathSciNet  Google Scholar 

  11. Laghrib A, Ben-Loghfyry A, Hadri A, Hakim A (2018) A nonconvex fractional order variational model for multi-frame image super-resolution. Signal Process Image Commun 67:1–11

    Article  Google Scholar 

  12. Ben-Loghfyry A, Hakim A, Laghrib A (2023) A denoising model based on the fractional beltrami regularization and its numerical solution. J Appl Math Comput 69(2):1431–1463

    Article  MathSciNet  Google Scholar 

  13. Ben-Loghfyry A, Hakim A (2022) Time-fractional diffusion equation for signal and image smoothing. Math. Modeling and Comput 9(2):351–364

    Article  Google Scholar 

  14. Al-Mahdi A, Fairag F (2018) Block diagonal preconditioners for an image de-blurring problem with fractional total variation. In: Journal of Physics: Conference Series, vol 1132, p 012063. IOP Publishing

  15. Li Y, Qin J, Osher S, Liu W (2016) Graph fractional-order total variation eeg source reconstruction. In: 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp 101–104. IEEE

  16. Zhao W, Lu H (2017) Medical image fusion and denoising with alternating sequential filter and adaptive fractional order total variation. IEEE Trans Instrum Meas 66(9):2283–2294

    Article  ADS  Google Scholar 

  17. Zhang J, Chen K (2015) A total fractional-order variation model for image restoration with nonhomogeneous boundary conditions and its numerical solution. SIAM J Imaging Sci 8(4):2487–2518

    Article  MathSciNet  Google Scholar 

  18. Ben-Loghfyry A, Hakim A, Laghrib A (2022) A denoising model based on the fractional beltrami regularization and its numerical solution. J Appl Math Comput, 1–33

  19. Ben-loghfyry A, Hakim A (2022) Reaction-diffusion equation based on fractional-time anisotropic diffusion for textured images recovery. Int J Appl Comput Math 8(4):177

    Article  MathSciNet  Google Scholar 

  20. Hakim A, Ben-Loghfyry A (2019) A total variable-order variation model for image denoising. AIMS MATHEMATICS 4(5):1320–1335

    Article  MathSciNet  Google Scholar 

  21. Ben-loghfyry A, Hakim A (2023) Total variable-order variation as a regularizer applied on multi-frame image super-resolution. The Visual Computer, 1–11

  22. Baloochian H, Ghaffary HR, Balochian S (2017) Enhancing fingerprint image recognition algorithm using fractional derivative filters. Open Comput Sci 7(1):9–16

    Article  Google Scholar 

  23. Ferrah I, Chaou AK, Maadjoudj D, Teguar M (2020) Novel colour image encoding system combined with ann for discharges pattern recognition on polluted insulator model. IET Science, Measurement & Technology

  24. Nasreddine K, Benzinou A, Fablet R (2009) Signal and image registration: Application to decrypt marine biological archives. TRAITEMENT DU SIGNAL 26(4):255–268

    Google Scholar 

  25. Frohn-Schauf C, Henn S, Witsch K (2008) Multigrid based total variation image registration. Comput Vis Sci 11(2):101–113

    Article  MathSciNet  Google Scholar 

  26. Magiera J, Ray D, Hesthaven JS, Rohde C (2020) Constraint-aware neural networks for riemann problems. J Comput Phys 409:109345

    Article  MathSciNet  Google Scholar 

  27. Hintermüller M, Wu T (2015) Bilevel optimization for calibrating point spread functions in blind deconvolution

  28. Chen Y, Pock T (2016) Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE Trans Pattern Anal Mach Intell 39(6):1256–1272

    Article  PubMed  Google Scholar 

  29. Kunisch K, Pock T (2013) A bilevel optimization approach for parameter learning in variational models. SIAM J Imaging Sci 6(2):938–983

    Article  MathSciNet  Google Scholar 

  30. Chung J, Español MI, Nguyen T (2014) Optimal regularization parameters for general-form tikhonov regularization. arXiv preprint arXiv:1407.1911

  31. Ring W (2000) Structural properties of solutions to total variation regularization problems. ESAIM: Math Model Numer Anal 34(4):799–810

  32. Ben-loghfyry A, Hakim A (2023) Caputo fractional-time of a modified cahn-hilliard equation for the inpainting of binary images. Journal of Mathematical Modeling

  33. Song Y, Ermon S (2019) Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems 32

  34. Bilgic B, Chatnuntawech I, Fan AP, Setsompop K, Cauley SF, Wald LL, Adalsteinsson E (2014) Fast image reconstruction with l2-regularization. J Magn Reson Imaging 40(1):181–191

    Article  PubMed  Google Scholar 

  35. Javed SG, Majid A, Mirza AM, Khan A (2016) Multi-denoising based impulse noise removal from images using robust statistical features and genetic programming. Multimed Tools Appl 75:5887–5916

    Article  Google Scholar 

  36. Tian C, Zheng M, Zuo W, Zhang B, Zhang Y, Zhang D (2023) Multi-stage image denoising with the wavelet transform. Pattern Recogn 134:109050

    Article  Google Scholar 

  37. Wu Q (2023) Research on deep learning image processing technology of second-order partial differential equations. Neural Comput & Applic 35(3):2183–2195

    Article  Google Scholar 

  38. Ben-loghfyry A, Hakim A (2022) Robust time-fractional diffusion filtering for noise removal. Math Meth Appl Sci 45(16):9719–9735

    Article  MathSciNet  Google Scholar 

  39. Zhang C, Zhang Z, Feng Z, Yi L (2023) Joint sparse model with coupled dictionary for medical image fusion. Biomed Signal Process Control 79:104030

  40. Tabassum S, Gowre SC (2023) Optimal image denoising using patch-based convolutional neural network architecture. Multimedia Tools and Applications, 1–17

  41. Liu X, Hao C, Su Z, Qi Z, Fu S, Li Y, Han H (2023) Image inpainting algorithm based on tensor decomposition and weighted nuclear norm. Multimedia Tools and Applications 82(3):3433–3458

    Article  Google Scholar 

  42. Zhang T-T, Shu H, Lam K-Y, Chow C-Y, Li A (2023) Feature decomposition and enhancement for unsupervised medical ultrasound image denoising and instance segmentation. Appl Intell 53(8):9548–9561

    Article  Google Scholar 

  43. Sui J, Ma X, Zhang X, Pun M-O (2023) Gcrdn: Global context-driven residual dense network for remote sensing image super-resolution. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing

  44. Shi Y, Huang Z, Huang Z, Hua X, Hong H, Li L (2023) Hinrdnet: A half instance normalization residual dense network for passive millimetre wave image restoration. Infrared Physics & Technology 132:104722

    Article  Google Scholar 

  45. Liang J, Cao J, Sun G, Zhang K, Van Gool L, Timofte R (2021) Swinir: Image restoration using swin transformer. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp 1833–1844

  46. Zhang Y, Tian Y, Kong Y, Zhong B, Fu Y (2020) Residual dense network for image restoration. IEEE Trans Pattern Anal Mach Intell 43(7):2480–2495

    Article  Google Scholar 

  47. Singh H, Ahmed AS, Melandsø F, Habib A (2023) Ultrasonic image denoising using machine learning in point contact excitation and detection method. Ultrasonics 127:106834

    Article  PubMed  Google Scholar 

  48. Vaksman G, Elad M (2023) Patch-craft self-supervised training for correlated image denoising. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 5795–5804

  49. Liu Y, Qin Z, Anwar S, Ji P, Kim D, Caldwell S, Gedeon T (2021) Invertible denoising network: A light solution for real noise removal. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 13365–13374

  50. Kulikov V, Yadin S, Kleiner M, Michaeli T (2023) Sinddm: A single image denoising diffusion model. In: International Conference on Machine Learning, pp. 17920–17930. PMLR

  51. Arun P, Sahare SA, Gopi VP (2023) Rescownet: A deep convolutional neural network with residual learning based on dt-cwt for despeckling optical coherence tomography images. Optik 284:170924

    Article  Google Scholar 

  52. Stergiopoulou V, Mukherjee S, Calatroni L, Blanc-Féraud L (2023) Fluctuation-based deconvolution in fluorescence microscopy using plug-and-play denoisers. In: International Conference on Scale Space and Variational Methods in Computer Vision, pp 498–510. Springer

  53. Liu D, Wang J, Shan Q, Smyl D, Deng J, Du J (2023)Deepeit: deep image prior enabled electrical impedance tomography. IEEE Transactions on Pattern Analysis and Machine Intelligence

  54. Chen W, Wipf D, Rodrigues M (2021) Deep learning for linear inverse problems using the plug-and-play priors framework. In: ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp 8098–8102. IEEE

  55. Mazé F, Ahmed F (2023) Diffusion models beat gans on topology optimization. In: Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), Washington, DC

  56. Ho J, Saharia C, Chan W, Fleet DJ, Norouzi M, Salimans T (2022) Cascaded diffusion models for high fidelity image generation. J Mach Learn Res 23(1):2249–2281

    MathSciNet  Google Scholar 

  57. Kawar B, Ganz R, Elad M (2022) Enhancing diffusion-based image synthesis with robust classifier guidance. arXiv preprint arXiv:2208.08664

  58. Elad M (2002) On the origin of the bilateral filter and ways to improve it. IEEE Trans Image Process 11(10):1141–1151

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  59. Bredies K, Kunisch K, Pock T (2010) Total generalized variation. SIAM J Imaging Sci 3(3):492–526

    Article  MathSciNet  Google Scholar 

  60. Zhang K, Zuo W, Chen Y, Meng D, Zhang L (2017) Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Trans Image Process 26(7):3142–3155

    Article  ADS  MathSciNet  PubMed  Google Scholar 

  61. Zhang K, Zuo W, Zhang L (2018) Ffdnet: Toward a fast and flexible solution for cnn-based image denoising. IEEE Trans Image Process 27(9):4608–4622

    Article  ADS  MathSciNet  Google Scholar 

  62. Li Y, Zhang J, Wang M (2017) Improved bm3d denoising method. IET Image Process 11(12):1197–1204

    Article  Google Scholar 

Download references

Acknowledgements

We would like to thank the editor and anonymous reviewers for their helpful comments and valuable suggestions.

Author information

Authors and Affiliations

  1. Department of mathematics, Faculty of Sciences and Technologies Mohammedia, University Hassan II, Casablanca, Morocco

    Anouar Ben-loghfyry

  2. LAMAI FST, University of Cadi Ayyad, Marrakesh, Morocco

    Abdelilah Hakim

Authors
  1. Anouar Ben-loghfyry
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abdelilah Hakim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Anouar Ben-loghfyry.

Ethics declarations

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ben-loghfyry, A., Hakim, A. A bilevel optimization problem with deep learning based on fractional total variation for image denoising. Multimed Tools Appl 83, 28595–28614 (2024). https://doi.org/10.1007/s11042-023-16583-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-16583-4

Keywords

Search

Navigation