Skip to main content
SpringerLink
Log in

Efficient video cutout based on adaptive multilevel banded method

  • Research Paper
  • Special Focus
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Although many efficient image cutout approaches have been proposed, it is still a challenge to cut out foreground objects from a video clip because of the costs of time and memory consumption and the difficulty in maintaining temporal coherence for the cutout results. In this paper, we propose a novel and efficient video cutout approach by extending a multilevel banded image segmentation method to video cutout. In our method, the segmentation results are propagated frame by frame at the lowest resolution level based on optical flow, and are refined in image pyramids level by level through banded graph cuts. We construct an adaptive narrow band to ensure that the segmentation result falls exactly into the narrow band during propagation. Experimental results show that the video cutout approach based on adaptive multilevel bands dramatically reduces time and memory costs, and achieves ideal segmentation results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Rother C, Kolmogorov V, Blake A. “Grabcut” — interactive foreground extraction using iterated graph cut. In: Proceeding of ACM SIGGRAPH, 2004. 309–341

  2. Li Y, Sun J, Tang C K, et al. Lazy snapping. In: Proceeding of ACM SIGGRAPH, 2004. 303–308

  3. Liu J Y, Sun J, Shum H Y. Paint selection. In: Proceeding of ACM SIGGRAPH, 2009

  4. Huang H, Zhang L, Zhang H C. RepSnapping: efficient image cutout for repeated scene elements. In: Proceeding of Pacific Graphics, 2011

  5. Boykov Y Y, Jolly M P. Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. In: Proceeding of ICCV, 2001. 105–112

  6. Boykov Y Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Analysis Mach Intell, 2001, 23: 1222–1239

    Article  Google Scholar 

  7. Lombaert H, Sun Y Y. A multilevel banded graph cuts method for fast image segmentation. In: Proceeding of ICCV, 2005. 259–265

  8. Li Y, Sun J, Shum H Y. Video object cut and paste. In: Proceeding of ACM SIGGRAPH, 2005. 595–600

  9. Wang J, Bhat P, Agrawala M, Cohen M F. Interactive video cutout. In: Proceeding of ACM SIGGRAPH, 2005. 585–594

  10. Vincent L, Soille P. Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Trans Pattern Analysis Mach Intell, 1991, 13: 583–598

    Article  Google Scholar 

  11. Comaniciu D, Meer P. Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Analysis Mach Intell, 2002, 24: 603–619

    Article  Google Scholar 

  12. Lucas B D, Kanade T. An iterative image registration technique with an application to stereo vision. In: Proceeding of Imaging Understanding Workshop, 1981. 121–130

  13. Chuang Y J, Agarwala A, Curless B, et al. Video matting of complex scenes. In: Proceeding of ACM SIGGRAPH, 2002. 243–248

  14. Bai X, Wang J, Simons H D, et al. Video SnapCut: robust video object cutout using localized classifiers. In: Proceeding of ACM SIGGRAPH, 2009

  15. Xie Z F, Shen Y, Ma L Z, et al. Seamless video composition using optimized mean-value cloning. Visual Computer, 2010, 26: 1123–1134

    Article  Google Scholar 

  16. Jia Y T, Hu S M, Martin R R. Video completion using tracking and fragment merging. Visual Computer, 2005, 21: 601–610

    Article  Google Scholar 

  17. Criminisi A, Cross G, Blake A, et al. Bilayer segmentation of live video. In: Proceeding of CVPR, 2006. 53–60

  18. Zhang G F, Jia J Y, Hua W, et al. Robust bilayer segmentation and motion/depth estimation with a handheld camera. TPAMI, 2011, 33: 603–617

    Article  MATH  Google Scholar 

  19. Grundmann M, Kwatra V, Han M, et al. Efficient hierarchical graph-based video segmentation. In: Proceeding of CVPR, 2010

  20. Zhang S H, Chen T, Zhang Y F, et al. Vectorizing Cartoon Animations. IEEE Trans Visual Computer Graph, 2009, 15: 618–629

    Article  Google Scholar 

  21. Kopf J, Cohen M F, Lischinski D, et al. Joint bilateral upsampling. In: Proceeding of ACM SIGGRAPH, 2007

  22. Bai X, Sapiro G. A geodesic framework for fast interactive image and video segmentation and matting. In: Proceeding of ICCV, 2007. 1038–1044

  23. Cheng M M, Zhang G X, Mitra N J, et al. Global contrast based salient region detection. In: Proceeding of CVPR, 2011. 409–416

  24. Chen T, Cheng M M, Tan P, et al. Sketch2Photo: internet image montage. ACM Trans Graph, 2009, 28: 1–10

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    MinGang Chen, BoCong Sui & LiZhuang Ma

  2. Software Engineering Institute, East China Normal University, Shanghai, 200062, China

    Yan Gao

Authors
  1. MinGang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. BoCong Sui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. LiZhuang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to MinGang Chen.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Chen, M., Sui, B., Gao, Y. et al. Efficient video cutout based on adaptive multilevel banded method. Sci. China Inf. Sci. 55, 1082–1092 (2012). https://doi.org/10.1007/s11432-012-4560-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-012-4560-4

Keywords

Search

Navigation