Abstract
This paper presents an algorithm for image completion based on the views of large displacement. A distinct from most existing image completion methods, which exploit only the target image’s own information to complete the damaged regions, our algorithm makes full use of a large displacement view (LDV) of the same scene, which introduces enough information to resolve the original ill-posed problem. To eliminate any perspective distortion during the warping of the LDV image, we first decompose the target image and the LDV one into several corresponding planar scene regions (PSRs) and transform the candidate PSRs on the LDV image onto their correspondences on the target image. Then using the transformed PSRs, we develop a new image repairing algorithm, coupled with graph cut based image stitching, texture synthesis based image inpainting, and image fusion based hole filling, to complete the missing regions seamlessly. Finally, the ghost effect between the repaired region and its surroundings is eliminated by Poisson image blending. Our algorithm effectively preserves the structure information on the missing area of the target image and produces a repaired result comparable to its original appearance. Experiments show the effectiveness of our method.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ballester, C., Caselles, V., Verdera, J., Bertalmio, M., Sapiro, G.: A variational model for filling-in gray level and color images. In: Proceedings of the IEEE ICCV 2001. vol. 1, pp. 10–16 (2001)
Bertalmio, M., Bertozzi A.L., Sapiro, G.: Navier–Stokes, fluid dynamics, and image and video inpainting. In: Proceedings of the IEEE CVPR 2001, vol. I, pp. 355–362 (2001)
Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C.: Image inpainting. In: Proceedings of ACM SIGGRAPH 2000, pp. 417–424. New Orleans, Louisiana (2000)
Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. IEEE Trans. Pattern Anal. Mach. Intell. 26(9), 1124–1137 (2004)
Chan, T., Shen, J.H.: Variational image inpainting. Commun. Pure Appl. Math. 58(5), 579–619 (2005)
Collis, B., Kokaram, A.: Filling in the gaps. IEE Electron. Systems Software 2(4), 22–28 (2004)
Comaniciu, D., Meer, P.: Mean Shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002)
Criminisi, A., Pérez, P., Toyama, K.: Region filling and object removal by exemplar-based image inpainting. IEEE Trans. Image Processing 13(9), 1200–1212 (2004)
Drori, I., Cohen-Or, D., Yeshurum, H.: Fragment-based image completion. ACM Trans. Graph. 22(3), 303–312 (2003)
Fadili, M.J., Starck, J.L.: EM algorithm for sparse representation-based image inpainting. In: Proceedings of the IEEE ICIP 2005, vol. 2, pp. 61–64 (2005)
Hartley, R., Zisserman, A.: Multiple view geometry in computer vision. Cambridge University Press, (2000) (ISBN: 0521623049)
Jia, J., Tai, Y.W., Wu, T.P., Tang, C.K.: Video repairing under variable illumination using cyclic motions. IEEE Trans. Pattern Anal. Mach. Intell. 28(5), 832–839 (2006)
Jia, J.Y., Tang, C.K.: Image repairing: robust image synthesis by adaptive ND tensor voting. In: Proceedings of the IEEE CVPR 2003. vol. 1, pp. 643–650 (2003)
Jia, Y.T., Hu, S.M., Martin, R.R.: Video completion using tracking and fragment merging. Visual Comput. (PG 2005) 21(8–10), 601–610 (2005)
Komodakis, N., Tziritas, G.: Image completion using global optimization. In: Proceedings of the IEEE CVPR 2006, vol. 1, pp. 442–452 (2006)
Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A.: Graphcut textures: image and video synthesis using graph cuts. ACM Trans. Graph. 22(3), 277–286 (2003)
Levin, A., Zomet, A., Weiss, Y.: Learning how to inpaint from global image statistics. In: Proceedings of the IEEE ICCV 2003, pp. 305–312. Nice, France (2003)
Lowe, D.G.: Distinctive image features from scale-invariant interest points. Int. J. Comput. Vision 60(2), 91–110 (2004)
Pavić, D., Schönefeld, V., Kobbelt, L.: Interactive image completion with perspective correction. Visual Comput. (PG 2006) 22(9–11), 671–681 (2006)
Pérez, P., Gangnet, M., Blake, A.: Poisson image editing. ACM Trans. Graph. 22(3), 313–318 (2003)
Shen, J.H.: Inpainting and the fundamental problem of image processing. SIAM News 36(5), 1–4 (2003)
Shih, T.K., Chang, R.C.: Digital inpainting – survey and multilayer image inpainting algorithms. In: Proceedings of ICITA 2005, pp. 15–24 (2005)
Shiratori, T., Matsushita, Y., Kang, S.B., Tang, X.: Video completion by motion field transfer. In: Proceedings of the IEEE CVPR 2006, vol. 1, pp. 411–418 (2006)
Sun, J., Lu, Y., Jia, J.Y., Shum, H.Y.: Image completion with structure propagation. ACM Trans. Graph. 24(3), 861–868 (2005)
Szeliski, R.: Image alignment and stitching: a tutorial. In: Microsoft Research Technical Reports, http://research.microsoft.com/vision/visionbasedmodeling/publications/MSR-TR-2004-92-Jan26.pdf. (2004)
Wexler, Y., Shechtman, E., Irani, M.: Space-time video completion. In: Proceedings of the IEEE CVPR 2004, vol. 1, pp. 120–127 (2004)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Liu, C., Guo, Y., Pan, L. et al. Image completion based on views of large displacement. Visual Comput 23, 833–841 (2007). https://doi.org/10.1007/s00371-007-0137-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00371-007-0137-4