Skip to main content
Log in

Exemplar-based video inpainting with large patches

  • Published:
Journal of Zhejiang University SCIENCE C Aims and scope Submit manuscript

Abstract

Inpainting is the process of reconstructing damaged regions of images and video frames. This study deals with weaknesses of the current video inpainting techniques, when an object is totally damaged, and a framework for video inpainting is proposed. Using this framework, the moving object is separated from the background. A large mosaic image is constructed using the moving object and then a patch-based method with large patches is used to fill holes. In each frame, the inpainted foreground is obtained by placing the object in its location. Missing areas of the stationary background are also filled separately and the final video is produced by composing the inpainted background and object frames. Results for three video sequences with an occluded object show that this approach represents the object in the missing region better than other approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  • Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C., 2000. Image Inpainting. Proc. ACM SIGGRAPH Conf. on Computer Graphics, p.417–424. [doi:10.1145/344779.344972]

  • Bertalmio, M., Bertozzi, A.L., Sapiro, G., 2001. Navier-Stokes, Fluid Dynamics, and Image and Video Inpainting. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 1:355–362. [doi:10.1109/CVPR.2001.990497]

    Google Scholar 

  • Bertalmio, M., Vese, L., Sapiro, G., Osher, S., 2003. Simultaneous structure and texture image inpainting. IEEE Trans. Image Process., 12(8):882–889. [doi:10.1109/TIP.2003.815261]

    Article  Google Scholar 

  • Cheung, S., Zhao, J., Venkatesh, M.V., 2006. Efficient Object-Based Video Inpainting. Proc. IEEE Int. Conf. on Image Processing, p.705–708. [doi:10.1109/ICIP.2006.312432]

  • Criminisi, A., Perez, P., Toyama, K., 2004. Region filling and object removal by exemplar-based inpainting. IEEE Trans. Image Process., 13(9):1200–1212. [doi:10.1109/TIP.2004.833105]

    Article  Google Scholar 

  • Ho, H.T., Goecke, R., 2007. Automatic Parametrisation for an Image Completion Method Based on Markov Random Fields. Proc. IEEE Int. Conf. on Image Processing, 3:541–544. [doi:10.1109/ICIP.2007.4379366]

    Google Scholar 

  • Liu, D., Sun, X., Wu, F., Li, S., Zhang, Y.Q., 2007. Image compression with edge-based inpainting. IEEE Trans. Circ. Syst. Video Technol., 17(10):1273–1287. [doi:10.1109/TCSVT.2007.903663]

    Article  Google Scholar 

  • Matsushita, Y., Ofek, E., Ge, W., Tang, X., Shum, H.Y., 2006. Full-frame video stabilization with motion inpainting. IEEE Trans. Pattern Anal. Mach. Intell., 28(7):1150–1163. [doi:10.1109/TPAMI.2006.141]

    Article  Google Scholar 

  • Oliveira, M.M., Bowen, B., McKenna, R., Chang, Y.S., 2001. Fast Digital Image Inpainting. Proc. Int. Conf. on Visualization, Imaging and Image Processing, p.261–266.

  • Patwardhan, K.A., Sapiro, G., Bertalmio, M., 2007. Video inpainting under constrained camera motion. IEEE Trans. Image Process., 16(2):545–553. [doi:10.1109/TIP.2006.888343]

    Article  MathSciNet  Google Scholar 

  • Sheikh, Y., Shah, M., 2005. Bayesian Object Detection in Dynamic Scenes. Proc. IEEE Conf. on Computer Vision and Pattern Recognition, 1:74–79. [doi:10.1109/CVPR.2005.86]

    Google Scholar 

  • Shen, Y., Lu, F., Cao, X., Foroosh, H., 2006. Video Completion for Perspective Camera under Constrained Motion. Proc. 18th Int. Conf. on Pattern Recognition, 3:63–66. [doi:10.1109/ICPR.2006.1169]

    Google Scholar 

  • Stauffer, C., Grimson, W.E.L., 1999. Adaptive Background Mixture Models for Real-Time Tracking. Proc. Computer Vision and Pattern Recognition, p.246–252. [doi:10.1109/CVPR.1999.784637]

  • Sun, J., Yuan, L., Jia, J., Shum, H.Y., 2005. Image completion with structure propagation. ACM Trans. Graph., 24(3): 861–868. [doi:10.1145/1073204.1073274]

    Article  Google Scholar 

  • Venkatesh, M.V., Cheung, S.S., Zhao, J., 2009. Efficient object-based video inpainting. Pattern Recogn. Lett., 30(2): 168–179. [doi:10.1016/j.patrec.2008.03.011]

    Article  Google Scholar 

  • Wang, H., Li, H., Li, B., 2007. Video Inpainting for Largely Occluded Moving Human. Proc. IEEE Int. Conf. on Multimedia and Expo, p.1719–1722. [doi:10.1109/ICME.2007.4285001]

  • Wexler, Y., Shechtman, E., Irani, M., 2004. Space-Time Video Completion. Proc. IEEE Computer Society Conf. on Computer Vision and Pattern Recognition, 1:120–127. [doi:10.1109/CVPR.2004.1315022]

    Google Scholar 

  • Wexler, Y., Shechtman, E., Irani, M., 2007. Space-time completion of video. IEEE Trans. Pattern Anal. Mach. Intell., 29(3):463–476. [doi:10.1109/TPAMI.2007.60]

    Article  Google Scholar 

  • Wren, C.R., Azarbayejani, A., Darrell, T., Pentland, A.P., 1997. Pfinder: Real-time tracking of the human body. IEEE Trans. Pattern Anal. Mach. Intell., 19(7):780–785. [doi:10.1109/34.598236]

    Article  Google Scholar 

  • Zhang, Y., Xiao, J., Shah, M., 2005. Motion Layer Based Object Removal in Videos. Proc. IEEE Workshop on Applications of Computer Vision, p.516–521. [doi:10.1109/ACVMOT.2005.75]

  • Zivkovic, Z., 2004. Improved Adaptive Gaussian Mixture Model for Background Subtraction. Proc. 17th Int. Conf. on Pattern Recognition, 2:28–31. [doi:10.1109/ICPR.2004.1333992]

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Abbas Koochari.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Koochari, A., Soryani, M. Exemplar-based video inpainting with large patches. J. Zhejiang Univ. - Sci. C 11, 270–277 (2010). https://doi.org/10.1631/jzus.C0910308

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1631/jzus.C0910308

Key words

CLC number

Navigation