Skip to main content
SpringerLink
Log in

Harmonic image inpainting using the charge simulation method

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

It was recently reported that harmonic inpainting or Laplace interpolation when used in the context of image compression can yield impressive reconstruction results if the encoded pixels were carefully selected. Mathematically, the problem translates into a mixed Dirichlet–Neumann boundary value problem with Dirichlet data coming from the known observations and reflecting conditions being imposed on the image physical boundary. Classical numerical solutions depend on finite difference schemes, which often induce instabilities and rely heavily on the choice of a convenient regularization parameter. In this paper, we propose an alternative numerical method, which is able to provide a robust harmonic reconstruction without requiring neither numerical integration nor discretization of the inpainting domain or its boundary. In fact, our approach is connected with the charge simulation method powered with the fast multipole method. Thereby, we approximate the harmonic reconstruction by a linear combination of the fundamental solutions of the Laplace equation. The experimental results on standard test images using uniformly, randomly and optimally distributed masks of different densities demonstrate the superior performance of our numerical approach over the finite difference method in terms of both reconstruction quality and speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bertalmio M, Sapiro G, Caselles V, Ballester C (2000) Image inpainting. In: Proceedings of the 27th annual conference on computer graphics and interactive techniques, pp 417–424

  2. Galić I, Weickert J, Welk M, Bruhn A, Belyaev A, Seidel H-P (2008) Image compression with anisotropic diffusion. J Math Imaging Vis 31(2–3):255–269

    Article  MathSciNet  Google Scholar 

  3. Schönlieb C-B (2015) Partial differential equation methods for image inpainting, vol 29. Cambridge University Press, Cambridge

    Book  Google Scholar 

  4. Caselles V, Morel J-M, Sbert C (1998) An axiomatic approach to image interpolation. IEEE Trans Image Process 7(3):376–386

    Article  MathSciNet  Google Scholar 

  5. Shen J, Chan TF (2002) Mathematical models for local nontexture inpaintings. SIAM J Appl Math 62(3):1019–1043

    Article  MathSciNet  Google Scholar 

  6. Mainberger M, Hoffmann S, Weickert J, Tang CH, Johannsen D, Neumann F, Doerr B (2011) Optimising spatial and tonal data for homogeneous diffusion inpainting. In: International conference on scale space and variational methods in computer vision. Springer, pp 26–37

  7. Carlsson S (1988) Sketch based coding of grey level images. Signal Process 15(1):57–83

    Article  Google Scholar 

  8. Belhachmi Z, Bucur D, Burgeth B, Weickert J (2009) How to choose interpolation data in images. SIAM J Appl Math 70(1):333–352

    Article  MathSciNet  Google Scholar 

  9. Mainberger M, Bruhn A, Weickert J, Forchhammer S (2011) Edge-based compression of cartoon-like images with homogeneous diffusion. Pattern Recognit 44(9):1859–1873

    Article  Google Scholar 

  10. Hoeltgen L, Setzer S, Weickert J (2013) An optimal control approach to find sparse data for Laplace interpolation. In: International workshop on energy minimization methods in computer vision and pattern recognition. Springer, pp 151–164

  11. Chen Y, Ranftl R, Pock T (2014) A bi-level view of inpainting-based image compression. In: Computer vision winter workshop

  12. Gholami A, Hosseini SM (2013) A balanced combination of Tikhonov and total variation regularizations for reconstruction of piecewise-smooth signals. Signal Process 93(7):1945–1960

    Article  Google Scholar 

  13. Wei W, Zhou B, Połap D, Woźniak M (2019) A regional adaptive variational pde model for computed tomography image reconstruction. Pattern Recognit 92:64–81

    Article  Google Scholar 

  14. Kalmoun EM (2020) A new Tikhonov-TV regularization for optical flow computation. In: Pan Z, Wang X (eds) Eleventh international conference on graphics and image processing (ICGIP 2019), vol 11373. International Society for Optics and Photonics, SPIE, pp 644–648

  15. Distasi R, Nappi M, Vitulano S (1997) Image compression by b-tree triangular coding. IEEE Trans Commun 45(9):1095–1100

    Article  Google Scholar 

  16. Schmaltz C, Peter P, Mainberger M, Ebel F, Weickert J, Bruhn A (2014) Understanding, optimising, and extending data compression with anisotropic diffusion. Int J Comput Vis 108(3):222–240

    Article  MathSciNet  Google Scholar 

  17. Peter P, Hoffmann S, Nedwed F, Hoeltgen L, Weickert J (2016) Evaluating the true potential of diffusion-based inpainting in a compression context. Signal Process Image Commun 46:40–53

    Article  Google Scholar 

  18. Chan TF, Kang SH (2006) Error analysis for image inpainting. J Math Imaging Vis 26(1–2):85–103

    Article  MathSciNet  Google Scholar 

  19. Katsurada M (1998) A mathematical study of the charge simulation method by use of peripheral conformal mappings. Mem Inst Sci Tech Meiji Univ 37(8):195–212

    Google Scholar 

  20. Singer H, Steinbigler H, Weiss P (1974) A charge simulation method for the calculation of high voltage fields. IEEE Trans Power Apparatus Syst 5:1660–1668

    Article  Google Scholar 

  21. Augustin MA (2015) A method of fundamental solutions in poroelasticity to model the stress field in geothermal reservoirs. Springer, Heidelberg

    Book  Google Scholar 

  22. Wang H, Qin Q-H (2019) Methods of fundamental solutions in solid mechanics. Elsevier, Amsterdam

    Google Scholar 

  23. Cheng AHD, Hong Y (2020) An overview of the method of fundamental solutions—solvability, uniqueness, convergence, and stability. Eng Anal Bound Elem 120:118–152

    Article  MathSciNet  Google Scholar 

  24. Fairweather G, Karageorghis A (1998) The method of fundamental solutions for elliptic boundary value problems. Adv Comput Math 9:69–95

    Article  MathSciNet  Google Scholar 

  25. Greengard L, Rokhlin V (1987) A fast algorithm for particle simulations. J Comput Phys 73(2):325–348

    Article  MathSciNet  Google Scholar 

  26. Amano K (1998) A charge simulation method for numerical conformal mapping onto circular and radial slit domains. SIAM J Sci Comput 19:1169–1187

    Article  MathSciNet  Google Scholar 

  27. Okano D, Ogata H, Amano K (2003) A method of numerical conformal mapping of curved slit domains by the charge simulation method. J Comput Appl Math 152:441–450

    Article  MathSciNet  Google Scholar 

  28. Amano K, Okano D (2010) A circular and radial slit mapping of unbounded multiply connected domains. JSIAM Lett 2:53–56

    Article  MathSciNet  Google Scholar 

  29. Amano K, Okano D, Ogata H, Sugihara M (2003) Numerical conformal mappings of unbounded multiply-connected domains using the charge simulation method. Bull Malays Math Sci Soc 26:35–51

    MathSciNet  MATH  Google Scholar 

  30. Benzi M, Golub GH, Liesen J (2005) Numerical solution of saddle point problems. Acta Numerica 14:1–137

    Article  MathSciNet  Google Scholar 

  31. Wendland WL, Christiansen S (1986) On the condition number of the influence matrix belonging to some first kind integral equations with logarithmic kernel. Appl Anal 21(3):175–183

    Article  MathSciNet  Google Scholar 

  32. Hoffmann S, Plonka G, Weickert J (2015) Discrete Green’s functions for harmonic and biharmonic inpainting with sparse atoms. In: International workshop on energy minimization methods in computer vision and pattern recognition. Springer, pp 169–182

  33. Greengard L, Gimbutas Z (2012) FMMLIB2D: A MATLAB toolbox for fast multipole method in two dimensions, version 1.2. edition. http://www.cims.nyu.edu/cmcl/fmm2dlib/fmm2dlib.html. Accessed 1 Jan 2018

  34. Parisotto S, Schönlieb C (2020) Matlab codes for the image inpainting problem. GitHub repository, MATLAB Central File Exchange, September 2016. Retrieved September 23, 2020

Download references

Acknowledgements

The authors would like to thank Dr. Matthias Augustin and Sarah Andris from the Mathematical Image Analysis Group headed by Prof. Joachim Weickert at Saarland University for the fruitful discussion and for providing the images together with the optimal masks shown in Figs. 5, 6, 7 and 8.

Author information

Authors and Affiliations

  1. School of Science and Engineering, Al Akhawayn University in Ifrane, PO Box 104, Ifrane, 53000, Morocco

    El Mostafa Kalmoun

  2. Program of Mathematics, Department of Mathematics, Statistics and Physics, College of Arts and Sciences, Qatar University, P.O. Box 2713, Doha, Qatar

    Mohamed M. S. Nasser

Authors
  1. El Mostafa Kalmoun
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed M. S. Nasser
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to El Mostafa Kalmoun.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kalmoun, E.M., Nasser, M.M.S. Harmonic image inpainting using the charge simulation method. Pattern Anal Applic 25, 795–806 (2022). https://doi.org/10.1007/s10044-022-01074-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-022-01074-3

Keywords

Search

Navigation