Skip to main content
Springer Nature Link
Log in

Curvature-Dependent Elastic Bending Total Variation Model for Image Inpainting with the SAV Algorithm

  • Published:
Journal of Scientific Computing Aims and scope Submit manuscript

Abstract

Image inpainting is pivotal within the realm of image processing, and many efforts have been dedicated to modeling, theory, and numerical analysis in this research area. In this paper, we propose a curvature-dependent elastic bending total variation model for the inpainting problem, in which the elastic bending energy in the phase-field framework introduces geometric information and the total variation term maintains the sharpness of the inpainting edge, referred to as elastic bending-TV model. The energy stability is theoretically proved based on the scalar auxiliary variable method. Additionally, an adaptive time-stepping algorithm is used to further improve the computational efficiency. Numerical experiments illustrate the effectiveness of the proposed model and verify the capability of our model in image inpainting.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Data Availability

Enquiries about data availability should be directed to the authors.

References

  1. Ambrosio, L., Tortorelli, V.M.: Approximation of functional depending on jumps by elliptic functional via convergence. Commun. Pure Appl. Math. 43, 999–1036 (1990)

    MathSciNet  Google Scholar 

  2. Bai, X., Sun, J., Shen, J., Yao, W., Guo, Z.: A Ginzburg–Landau-\(H^{-1}\) model and its SAV algorithm for image inpainting. J. Sci. Comput. 96, 40 (2023)

    MathSciNet  Google Scholar 

  3. Belhachmi, Z., Kallel, M., Moakher, M., Theljani, A.: Weighted harmonic and complex Ginzburg-Landau equations for gray value image inpainting. Int. J. Numer. Anal. Mod. 13, 782–801 (2016)

    MathSciNet  Google Scholar 

  4. Bergmann, R., Weinmann, A.: A second-order TV-type approach for inpainting and denoising higer dimensional combined cyclic and vectoe space data. J. Math. Imaging Vis. 55, 401–427 (2016)

    Google Scholar 

  5. Bertozzi, A.L., Esedoglu, S., Gillette, A.: Inpainting of binary images using the Cahn–Hilliard equation. IEEE Trans. Images Process. 16, 285–291 (2006)

    MathSciNet  Google Scholar 

  6. Bertozzi, A.L., Schönlieb, C.B.: Unconditionally stable schemes for higher order inpainting. Commun. Math. Sci. 9, 413–457 (2011)

    MathSciNet  Google Scholar 

  7. Bertozzi, A., Esedoglu, S., Gillette, A.: Analysis of a two-scale Cahn–Hilliard model for binary image inpainting. Multiscale Model. Simul. 6, 913–936 (2007)

    MathSciNet  Google Scholar 

  8. Buyssens, P., Daisy, M., Tschumperlé, D., Lézoray, O.: Exemplar-based inpainting: Technical review and new heuristics for better geometric reconstructions. IEEE Trans. Image Process. 24, 1809–1824 (2015)

    MathSciNet  Google Scholar 

  9. Canham, P.B.: The minimum energy of bending as a possible explanation of the bicocave shape of the human red blood cell. J. Theor. Biol. 26, 77–81 (1970)

    Google Scholar 

  10. Cao, F., Gousseau, Y., Masnou, S., Pérez, P.: Geometrically guided examplar-based inpainting. SIAM J. Imaging Sci. 4, 1143–1170 (2011)

    MathSciNet  Google Scholar 

  11. Chan, T.F., Shen, J., Vese, L.: Variational PDE models in image processing. Not. Am. Math. Soc. 50, 14–26 (2002)

    MathSciNet  Google Scholar 

  12. Chan, T.F., Shen, J.: Image Processing and Analysis: Variational, PDE, Wavelet, and Stochastic Methods. Society for Industrial and Applied Mathematics, Philadelphia (2005)

    Google Scholar 

  13. Chambolle, A., Lions, P.-L.: Image recovery via total variation minimization and related problems. Numer. Math. 76, 167–188 (1997)

    MathSciNet  Google Scholar 

  14. Condat, L.: Discrete total variation: new definition and minimization. SIAM J. Imaging Sci. 10, 1285–1290 (2017)

    MathSciNet  Google Scholar 

  15. Du, Q., Liu, C., Wang, X.: A phase field approach in the numerical study of the elastic bending energy for vesicle members. J. Comput. Phys. 198, 450–468 (2004)

    MathSciNet  Google Scholar 

  16. Du, Q., Liu, C., Ryham, R., Wang, X.: Phase field modeling of the spontaneous curvature effect in cell membranes. Commun. Pure Appl. Anal. 4, 537–548 (2005)

    MathSciNet  Google Scholar 

  17. Du, Q., Wang, X.: Convergence of numerical approximations to a phase field bending elasticity model of membrance deformations. Int. J. Numer. Anal. Mod. 4, 441–459 (2007)

    Google Scholar 

  18. Du, Q.: Phase field calculus, curvature-dependent energies, and vesicle membranes. Philos. Mag. 91, 165–181 (2011)

    Google Scholar 

  19. Du, X., Cho, D., Bui, T.D.: Image segmentation and inpainting using hierarchical level set and texture mapping. Signal Process. 91, 852–863 (2011)

    Google Scholar 

  20. Evans, E.A.: Bending resistance and chemically induced moments in membrane bilayers. Biophys. J . 14, 923–931 (1974)

    Google Scholar 

  21. Esedoglu, S., Shen, J.: Digital inpainting based on the Mumford–Shah–Euler image model. Eur. J. Appl. Math. 13, 353–370 (2002)

    MathSciNet  Google Scholar 

  22. He, F., Wang, X., Chen, X.: A penalty relaxation method for image processing using Euler’s elastica model. SIAM J. Imaging Sci. 14, 389–417 (2021)

    MathSciNet  Google Scholar 

  23. Helfrich, W.: Elastic properties of lipid bilayers: theory and possible experiments. Z. Naturforsch. C 28, 693–703 (1973)

    Google Scholar 

  24. Kumar, B.V.R., Halim, A.: A linear fourth-order PDE-based gray-scale image inpainting model. Comput. Appl. Math. 38, 1–21 (2019)

    MathSciNet  Google Scholar 

  25. Liu, C., Qiao, Z., Zhang, Q.: Two-phase segmentation for intensity inhomogeneous images by the Allen–Cahn local binary fitting model. SIAM J. Sci. Comput. 44, B177–B196 (2022)

    MathSciNet  Google Scholar 

  26. Liu, C., Qiao, Z., Zhang, Q.: An active contour model with local variance force term and its efficient minimization solver for multiphase image segmentation. SIAM J. Imaging Sci. 16, 144–168 (2023)

    MathSciNet  Google Scholar 

  27. Liu, C., Qiao, Z., Zhang, Q.: Multi-phase image segmentation by the Allen–Cahn Chan–Vese model. Comput. Math. Appl. 141, 207–220 (2023)

    MathSciNet  Google Scholar 

  28. Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17, 53–69 (2008)

    MathSciNet  Google Scholar 

  29. Mumford, D.: Elastica and computer vision. In: Bajaj, C.L. (ed.) Algebraic Geometry and Its Applications, pp. 491–506. Springer, New York (1994)

    Google Scholar 

  30. Novak, A., Reinić, N.: Shock filter as the classifier for image inpainting problem using the Cahn–Hilliard equation. Comput. Math. Appl. 123, 105–114 (2022)

    MathSciNet  Google Scholar 

  31. Pratap, A., Sardana, N.: Machine learning-based image processing in the materials science and engineering: a review. Mater. Today Proc. 62, 7341–7347 (2022)

    Google Scholar 

  32. Qiao, Z., Zhang, Z., Tang, T.: An adaptive time-stepping strategy for the molecular beam epitaxy models. SIAM J. Sci. Comput. 33, 1395–1414 (2011)

    MathSciNet  Google Scholar 

  33. Qiao, Z., Zhang, Q.: Two-Phase image segmentation by the Allen–Cahn equation and a nonlocal edge detection operator. Numer. Math. Theory Methods Appl. 15, 1147–1172 (2022)

    MathSciNet  Google Scholar 

  34. Qiao, Q.: Image processing technology based on machine learning. IEEE Consum. Electron. Mag. 13(4), 90–99 (2022)

  35. Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D 60, 259–268 (1992)

    MathSciNet  Google Scholar 

  36. Seifert, U.: Adhesion of vesicles in two dimensions. Phys. Rev. A 43, 6803 (1991)

    MathSciNet  Google Scholar 

  37. Shen, J., Chan, T.F.: Mathematical models for local nontexture inpaintings. SIAM J. Appl. Math. 62, 1019–1043 (2002)

    MathSciNet  Google Scholar 

  38. Shen, J., Kang, S.H., Chan, T.F.: Euler’s elastica and curvature-based inpainting. SIAM J. Appl. Math. 62, 564–592 (2003)

    MathSciNet  Google Scholar 

  39. Shen, B., Hu, W., Zhang, Y., Zhang, Y.-J.: Image inpainting via sparse representation. In: Proceedings of 34th IEEE International Conference on Acoustics, Speech, and Signal Processing, pp. 697–700 (2009)

  40. Shen, J., Xu, J., Yang, J.: The scalar auxiliary variable (SAV) approach for gradient flows. J. Comput. Phys. 353, 407–416 (2018)

    MathSciNet  Google Scholar 

  41. Shen, J., Xu, J., Yang, J.: A new class of efficient and robust energy stable schemes for gradient flows. SIAM Rev. 61, 474–506 (2019)

    MathSciNet  Google Scholar 

  42. Shi, W., Feng, X.-Q., Gao, H.: Two-dimensional model of vesicle adhesion on curved substrates. Acta. Mech. Sin. 22, 529–535 (2006)

    Google Scholar 

  43. Söderlind, G.: Automatic control and adaptive time-stepping. Numer. Algorithms 31, 281–310 (2002)

    MathSciNet  Google Scholar 

  44. Theljani, A., Belhachmi, Z., Moakher, M.: High-order anisotropic diffusion operators in spaces of variable exponents and application to image inpainting and restoration problems. Nonlinear Anal. RWA 47, 251–271 (2019)

    MathSciNet  Google Scholar 

  45. Wang, X., Ju, L., Du, Q.: Efficient and stable exponential time differencing Runge–Kutta methods for phase field elastic bending energy models. J. Comput. Phys. 316, 21–38 (2016)

    MathSciNet  Google Scholar 

  46. Wang, D., Wang, X.P.: The iterative convolution-thresholding method (ICTM) for image segmentation. Pattern Recognit. 130, 108794 (2022)

    Google Scholar 

  47. Yang, X., Ju, L.: Efficient linear schemes with unconditional energy stability for the phase field elastic bending energy model. Comput. Methods Appl. Mech. Eng. 315, 691–721 (2017)

    MathSciNet  Google Scholar 

  48. Yang, X.: A novel fully-decoupled, second-order time-accurate, unconditionally energy stable scheme for a flow-coupled volume-conserved phased-filed elastic bending energy model. J. Comput. Phys. 432, 110015 (2021)

    Google Scholar 

  49. Yang, W., Huang, Z., Zhu, W.: Image segmentation using the Cahn–Hilliard equation. J. Sci. Comput. 79, 1057–1077 (2019)

    MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Chaoyu Liu at University of Cambridge for many valuable discussions and comments. We thank Prof. Wenjuan Yao at Harbin Institute of Technology for providing the MATLAB codes of [2], Dr. Fang He at Shenzhen Technology University for providing the MATLAB codes of [22], and Prof. Andrej Novak at University of Zagreb for providing the MATLAB codes of [30].

Funding

C. Nan’s work is partially supported by the Hong Kong Research Grants Council grant 15303121 and the Hong Kong Polytechnic University Postdoctoral Research Fund 1-W261. Z. Qiao’s work is partially supported by the Hong Kong Research Grants Council (RFS Project No. RFS2021-5S03 and GRF project No. 15302122) and the Hong Kong Polytechnic University internal grant No. 1-9BCT.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong

    Caixia Nan & Zhonghua Qiao

  2. School of Science, Harbin Institute of Technology, Shenzhen, 518055, China

    Qian Zhang

Authors
  1. Caixia Nan
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Zhonghua Qiao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Qian Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Qian Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nan, C., Qiao, Z. & Zhang, Q. Curvature-Dependent Elastic Bending Total Variation Model for Image Inpainting with the SAV Algorithm. J Sci Comput 101, 29 (2024). https://doi.org/10.1007/s10915-024-02666-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10915-024-02666-3

Keywords

Mathematics Subject Classification