Skip to main content
SpringerLink
Log in

An image inpainting method using pLSA-based search space estimation

  • Original Paper
  • Published:
Machine Vision and Applications Aims and scope Submit manuscript

Abstract

In this paper, we present a novel exemplar-based image inpainting technique based on the local context measure of the target patch. Three main steps of the proposed method are determination of patch priority, the search space estimation for the candidate patches and the patch completion to fill in the unknown pixels of the target patch. In patch priority, we emphasize on the structure by the spatial relationship of neighborhood similar patches and kernel regression based local image structure. We find the search space, sub-regions of the entire source region similar to the region surrounding the target patch, to find the candidate patches. The said search space is estimated using probabilistic latent semantic analysis (pLSA). Last, we infer the unknown pixels of the target patch using pLSA-based context and histogram similarity measure between the target patch and the candidate patches. Experimental results are found to be good compared to the competitive methods and may be used for digital restoration of images of defective or damaged artifacts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

Notes

  1. http://www.cc.gatech.edu/~sooraj/inpainting/.

  2. http://lafarren.com/image-completer/.

  3. http://www.transformyourmoney.com/planning-a-road-trip.html.

  4. http://upload.wikimedia.org/wikipedia/en/1/12/Pappikondalu.jpg.

  5. http://www.baseball-wallpapers.com/Chicago-Cubs/.

  6. http://irelandwithkids.com/connemara-ocean-view/ireland-photo/.

  7. http://yokoya.naist.jp/research/inpainting/.

  8. http://graphics.cs.cmu.edu/projects/scene-completion/.

References

  1. Ashikhmin, M.: Synthesizing natural textures. In: Proc. of ACM Symp. on Interactive 3D graphics, pp. 217–226 (2001)

  2. Ballester, C., Bertalmio, M., Caselles, V.: Filling-in by joint interpolation of vector fields and gray levels. IEEE Trans. Image Process. 10(8), 1200–1211 (2001)

    Article  MATH  MathSciNet  Google Scholar 

  3. Barcelos, C., Batista, M.: Image restoration using digital inpainting and noise removal. Image Vis. Comput. 25, 61–69 (2007)

    Article  Google Scholar 

  4. Barnes, C., Shechtman, E., Finkelstein, A., Goldman, D.B.: Patch-match: a randomized correspondence algorithm for structural image editing. ACM Trans. Graph. 28(3), 1–11 (2009)

    Article  Google Scholar 

  5. Bertalmio, M.: Strong-continuation, contrast-invariant inpainting with a third-order optimal pde. IEEE Trans. Image Process. 15(7), 1934–1938 (2006)

    Article  Google Scholar 

  6. Bertalmio, M., Bertozzi, A.L., Sapiro, G.: Navier–strokes, fluid dynamics, and image and video inpainting. In: Proc. IEEE Computer Society Conf. Computer Vision and Pattern Recognition (2001)

  7. Bertalmio, M., Sapiro, G.: Image Inpainting. In: Proc. of the ACM SIGGRAPH Conf. on Computer Graphics[C], pp. 417–424. New York, USA (2000)

  8. Bertalmio, M., Vese, L., Sapiro, G., Osher, S.: Simultaneous structure and texture image inpainting. IEEE Trans. Image Process. 12, 882–889 (2003)

    Article  Google Scholar 

  9. Bornemann, F., Marz, T.: Fast image inpainting based on coherence transport. J. Math. Imag. Vis 28(3), 259–278 (2007)

    Article  MathSciNet  Google Scholar 

  10. Bosch, A., Zisserman, A., Muoz, X.: Scene classification using a hybrid generative/discriminative approach. IEEE Trans. Pattern Anal. Mach. Intel. 30(4), 712–727 (2008)

  11. Bosch, A., Zisserman, A., Muoz, X.: Scene classification via plsa. In: ECCV (4)’06, pp. 517–530 (2006)

  12. Bugeau, A., Bertalmio, M., Caselles, V.: A comprehensive framework for image inpainting. IEEE Trans. Image Process. 19(10), 2634–2645 (2010)

  13. Calvetti, D., Sgallari, F., Somersalo, E.: Image inpainting with structural bootstrap priors. Image Vis. Comput. 24, 782–793 (2006)

    Article  Google Scholar 

  14. Chan, T., Shen, J.: Non-texture inpainting By curvature-driven diffusions. J. Vis. Commun. Image Represent. 12(4), 436–449 (2001)

    Article  Google Scholar 

  15. Criminisi, A., Perez, P., Toyama., K.: Region Filling and Object Removal by Exemplar-Based inpainting. IEEE Trans. Image Process. 13(9), 1200–1212 (2004)

  16. Darabi, S., Shechtman, E., Barnes, C., Goldman, D.B., Sen, P.: Image melding: combining inconsistent images using patch-based synthesis. ACM Trans. Graph. (TOG) (Proc. SIGGRAPH 2012) 31(4), 82:1–82:10 (2012)

    Google Scholar 

  17. Drori, I., Cohen-Or, D., Yeshurun, H.: Fragment based image completion 22(2003), 303–312 (2003)

  18. Efros, A., Leung, T.: Texture synthesis by non-parametric sampling. In: Proceeding of IEEE International of Conference on Computer Vision, vol. 2, pp. 1033–1038 (1999)

  19. Elad, M., Starck, J.L., Querre, P., Donoho, D.L.: Simultaneous cartoon and texture image inpainting using morphological component analysis. Appl. Comput. Harmon. Anal. 19, 340–358 (2005)

    Article  MATH  MathSciNet  Google Scholar 

  20. Fadili, M.J., Starck, J.L., Murtagh, F.: Inpainting and zooming using sparse representations. Comput. J. 52(1), 64–79 (2009)

    Article  Google Scholar 

  21. Hays, J., Efros, A.: Scene completion using millions of photographs. Commun. ACM 51(10), 87–94 (2008)

    Article  Google Scholar 

  22. He, K., Sun, J.: Statistics of patch offsets for image completion. In: Proc. 12th Eur. Conf. Comput. Vis., pp. 16–29 (2012)

  23. Hofmann, T.: Probabilistic latent semantic indexing. In: Proc. SIGIR Conf. Research and Development in Information Retrieval (1998)

  24. Hofmann, T.: Unsupervised learning by probabilistic latent semantic analysis. Mach. Learn. 41(2), 177–196 (2001)

    Article  Google Scholar 

  25. Huang, J., Kumar, S.R., Mitra, M., Zhu, W.J., Zabih, R.: Image indexing using color correlograms. In: Proceedings of IEEE International Conference on CVPR, pp. 762–768 (1997)

  26. Jia, J., Tang, C.: Image repairing: Robust image synthesis by adaptive N-D tensor voting. In: Proc. CVPR, pp. 643–650 (2003)

  27. Komodakis, N., Tziritas, G.: Image completion using global optimization. In: Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition., pp. 442–452 (2006)

  28. Komodakis, N., Tziritas., G.: Image completion using efficient belief propagation via priority scheduling and dynamic pruning. IEEE Trans. Image Process. 16(11), 2649–2661 (2007)

  29. Lee, J., Lee, D.K., Park, R.H.: Robust exemplar-based inpainting algorithm using region segmentation. IEEE Trans. Consum. Electron. 58(2), 553–561 (2012)

    Article  Google Scholar 

  30. Liu, Y., Caselles, V.: Exemplar-based image inpainting using multiscale graph cuts. IEEE Trans. Image Process. 22(5), 1699–1711 (2013)

    Article  MathSciNet  Google Scholar 

  31. Meur, O.L., Ebdelli, M., Guillemot, C.: Hierarchical super-resolution-based inpainting. IEEE Trans. Image Process. 22(10), 3779–3790 (2013)

  32. Pritch, Y., Kav-Venaki, E., Peleg, S.: Shift-map image editing. In: Proc. IEEE Int. Conf. on Computer Vision, pp. 151–158 (2009)

  33. Purkait, P., Chanda, B.: Image upscaling using multiple dictionaries of natural image patches. In: Proc. ACCV (2012)

  34. Qin, Y., Wang, F.: A Curvature constraint Exemplar-based image inpainting. In: Int. Conf. on Image Processing and Signal Processing, pp. 263–267 (2010)

  35. Quelhas, P., Monay, F., Odobez, J., Gatica-Perez, D., Tuytelaars, T.: A thousand words in a scene. IEEE Trans. Pattern Anal. Mach. Intel. 29(9), 1575–1589 (2007)

    Article  Google Scholar 

  36. Quelhas, P., Monay, F., Odobez, J., Gatica-Perez, D., Tuytelaars, T., Gool, L.V.: Modeling scenes with local descriptors and latent aspects. In: Intl Conf. Computer Vision, pp. 883–890 (Oct. 2005)

  37. Shen, B., Hu, W., Zhang, Y., Zhang, Y.: Image inpainting via sparse representation. In: Proc. IEEE Int. Conf. Acoustics, Speech and Signal Processing, pp. 697–700 (2009)

  38. Sivic, J., Russell, B., A. Efros, A.Z., Freeman, W.: Discovering objects and their locations in images. In: Proc. Int’l Conf. Computer Vision, pp. 370–377 (2005)

  39. Sun, J., Yuan, L., Jia, J., Shum, H.: Image completion with structure propagation. In: Proc. SIGGRAPH2005, pp. 861–868 (2005)

  40. Takeda, H., Farsiu, S., Milanfar, P.: Kernel regression for image processing and reconstruction. IEEE Trans. Image Process. 16(2), 349–366 (2007)

    Article  MathSciNet  Google Scholar 

  41. Tschumperle, D.: Fast anisotropic smoothing of multi-valued images using curvature-preserving pdes. Int. J. Comput. Vis 68(1), 65–82 (2006)

    Article  Google Scholar 

  42. Verbeek, J., Triggs, B.: Region classification with markov field aspect models. In: Proc. CVPR, pp. 1–8 (2007)

  43. Wohlberg, B.: Inpainting by joint optimization of linear combinations of exemplars. IEEE Signal Process. Lett. 18(1), 75–78 (2011)

  44. Wong, A., Orchard, J.: A nonlocal-means approach to examplar-based inpainting. In: IEEE Int. Conf. Image Processing (2008)

  45. Wu, J., Ruan, Q.: Object Removal By Cross Isophotes Exemplar-based Inapinting. In: Proc. 18th Int. Conf. on Pattern Recognition, vol. 3, pp. 810–813 (2006)

  46. Xu, Z., Sun, J.: Image inpainting by patch propagation using patch sparsity. IEEE Trans. Image Process. 19(5), 1153–1165 (2010)

    Article  MathSciNet  Google Scholar 

  47. Zhang, Y., Xiao, J., Hays, J., Tan, P.: Framebreak: Dramatic image extrapolation by guided shift-maps. In: Proc. CVPR, pp. 1171–1178 (2013)

Download references

Author information

Authors and Affiliations

  1. ECSU, Indian Statistical Institute, Kolkata, India

    Mrinmoy Ghorai & Bhabatosh Chanda

Authors
  1. Mrinmoy Ghorai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bhabatosh Chanda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mrinmoy Ghorai.

Additional information

This work is partially supported by Department of Science and Technology, Government of India (NRDMS/11/1586/09/Phase-I/Project No. 9.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghorai, M., Chanda, B. An image inpainting method using pLSA-based search space estimation. Machine Vision and Applications 26, 69–87 (2015). https://doi.org/10.1007/s00138-014-0647-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00138-014-0647-9

Keywords

Search

Navigation