Skip to main content
SpringerLink
Log in

Efficient Video Cutout by Paint Selection

  • Regular Paper
  • Published:
Journal of Computer Science and Technology Aims and scope Submit manuscript

Abstract

Video cutout refers to extracting moving objects from videos, which is an important step in many video editing tasks. Recent algorithms have limitations in terms of efficiency, interaction style, and robustness. This paper presents a novel method for progressive video cutout with less user interaction and fast feedback. By exploring local and compact features, an optimization is constructed based on a graph model which establishes spatial and temporal relationship of neighboring patches in video frames. This optimization enables an efficient solution for progressive video cutout using graph cuts. Furthermore, a sampling-based method for temporally coherent matting is proposed to further refine video cutout results. Experiments demonstrate that our video cutout by paint selection is more intuitive and efficient for users than previous stroke-based methods, and thus could be put into practical use.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Chen T, Zhu J Y, Shamir A, Hu S M. Motion-aware gradient domain video composition. IEEE Transactions on Image Processing, 2013, 22(7): 2532-2544.

    Article  Google Scholar 

  2. Lu S P, Zhang S H, Wei J, Hu S M, Martin R R. Time-line editing of objects in video. IEEE Trans. Vis. Comput. Graph., 2013, 19(7): 1218-1227.

    Article  Google Scholar 

  3. Xu K, Li Y, Ju T, Hu S M, Liu T Q. Efficient affinity-based edit propagation using K-D tree. ACM Trans. Graph., 2009, 28(5): 118:1-118:6.

    Google Scholar 

  4. Ma L Q, Xu K. Efficient antialiased edit propagation for images and videos. Computers & Graphics, 2012, 36(8): 1005-1012.

    Article  Google Scholar 

  5. Liu J Y, Sun J, Shum H Y. Paint selection. ACM Trans. Graph., 2009, 28(3): 69:1-69:7.

    Google Scholar 

  6. Tong R F, Zhang Y, Ding M. Video brush: A novel interface for efficient video cutout. Comput. Graph. Forum, 2011, 30(7): 2049-2057.

    Article  Google Scholar 

  7. Hu S M, Chen T, Xu K, Cheng M M, Martin R R. Internet visual media processing: A survey with graphics and vision applications. The Visual Computer, 2013, 29(5): 393-405.

    Article  Google Scholar 

  8. Wang J, Cohen M F. Image and video matting: A survey. Foundations and Trends® in Computer Graphics and Vision, 2007, 3(2): 97-175.

    Article  Google Scholar 

  9. Agarwala A, Hertzmann A, Salesin D, Seitz S M. Keyframe-based tracking for rotoscoping and animation. ACM Trans. Graph., 2004, 23(3): 584-591.

    Article  Google Scholar 

  10. Bai X, Wang J, Simons D, Sapiro G. Video SnapCut: Robust video object cutout using localized classifiers. ACM Trans. Graph., 2009, 28(3): 70:1-70:11.

    Article  Google Scholar 

  11. Kolmogorov V, Zabih R. What energy functions can be minimized via graph cuts? IEEE Trans. Pattern Anal. Mach. Intell., 2004, 26(2): 147-159.

    Article  Google Scholar 

  12. Rother C, Kolmogorov V, Blake A. “Grabcut”: Interactive foreground extraction using iterated graph cuts. ACM Trans. Graph., 2004, 23(3): 309-314.

    Article  Google Scholar 

  13. Li Y, Sun J, Shum H Y. Video object cut and paste. ACM Trans. Graph., 2005, 24(3): 595-600.

    Article  Google Scholar 

  14. Wang J, Bhat P, Colburn A, Agrawala M, Cohen M F. Interactive video cutout. ACM Trans. Graph., 2005, 24(3): 585-594.

    Article  Google Scholar 

  15. Shahrian E, Price B, Cohen S, Rajan D. Temporally coherent and spatially accurate video matting. Comput. Graph. Forum, 2014, 33(2): 381-390.

    Article  Google Scholar 

  16. Ju J L, Wang J, Liu Y B, Wang H Q, Dai Q H. A progressive tri-level segmentation approach for topology-change-aware video matting. Comput. Graph. Forum, 2013, 32(7): 245-253.

    Article  Google Scholar 

  17. Zhong F, Qin X Y, Peng Q S, Meng X X. Discontinuity-aware video object cutout. ACM Trans. Graph., 2012, 31(6): 175:1-175:10.

    Article  Google Scholar 

  18. Comaniciu D, Meer P. Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell., 2002, 24(5): 603-619.

    Article  Google Scholar 

  19. Li Y, Sun J, Tang C K, Shum H Y. Lazy snapping. ACM Trans. Graph., 2004, 23(3): 303-308.

    Article  Google Scholar 

  20. Huang H, Zhang L, Zhang H C. RepSnapping: Efficient image cutout for repeated scene elements. Comput. Graph. Forum, 2011, 30(7): 2059-2066.

    Article  Google Scholar 

  21. Kopf J, Cohen M F, Lischinski D, Uyttendaele M. Joint bilateral upsampling. ACM Trans. Graph., 2007, 26(3): 96:1-96:5.

    Google Scholar 

  22. Lombaert H, Sun Y Y, Grady L, Xu C Y. A multilevel banded graph cuts method for fast image segmentation. In Proc. the 10th IEEE International Conference on Computer Vision, Oct. 2005, pp.259-265.

  23. Bai X, Wang J, Simons D. Towards temporally-coherent video matting. In Proc. the 5th MIRAGE, Oct. 2011, pp.63-74.

  24. He K M, Rhemann C, Rother C, Tang X O, Sun J. A global sampling method for alpha matting. In Proc. the 24th IEEE Conference on Computer Vision and Pattern Recognition, Jun. 2011, pp.2049-2056.

  25. Barnes C, Shechtman E, Finkelstein A, Goldman D B. PatchMatch: A randomized correspondence algorithm for structural image editing. ACM Trans. Graph., 2009, 28(3): 24:1-24:11.

    Article  Google Scholar 

  26. Zhang S H, Li X Y, Hu S M, Martin R R. Online video stream abstraction and stylization. IEEE Transactions on Multimedia, 2011, 13(6): 1286-1294.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Zhejiang Radio and TV Technology, Zhejiang University of Media and Communications, Hangzhou, 310018, China

    Yun Zhang

  2. College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

    Yan-Long Tang & Ke-Li Cheng

Authors
  1. Yun Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan-Long Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ke-Li Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yun Zhang.

Additional information

This work was supported by the National High Technology Research and Development 863 Program of China under Grant No. 2013AA013903, the Zhejiang Provincial Natural Science Foundation of China under Grant No. LY14F020050, and the National Basic Research 973 Program of China under Grant No. 2011CB302205.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Tang, YL. & Cheng, KL. Efficient Video Cutout by Paint Selection. J. Comput. Sci. Technol. 30, 467–477 (2015). https://doi.org/10.1007/s11390-015-1537-y

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11390-015-1537-y

Keywords

Search

Navigation