Skip to main content
SpringerLink
Log in

Interactive Segmentation of High-Resolution Video Content Using Temporally Coherent Superpixels and Graph Cut

  • Conference paper
Advances in Visual Computing (ISVC 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8887))

Included in the following conference series:

Abstract

Interactive video segmentation has become a popular topic in computer vision and computer graphics. Discrete optimization using maximum flow algorithms is one of the preferred techniques to perform interactive video segmentation. This paper extends pixel based graph cut approaches to overcome the problem of high memory requirements. The basic idea is to use a graph cut optimization framework on top of temporally coherent superpixels. While grouping spatially coherent pixels sharing similar color, these algorithms additionally exploit the temporal connections between those image regions. Thereby the number of variables in the optimization framework is severely reduced. The effectiveness of the proposed algorithm is shown quantitatively, qualitatively and through timing comparisons of different temporally coherent superpixel approaches. Experiments on video sequences show that temporally coherent superpixels lead to significant speed-up and reduced memory consumption. Thus, video sequences can be interactively segmented in a more efficient manner while producing better segmentation quality when compared to other approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Wang, S., Lu, H., Yang, F., Yang, M.H.: Superpixel tracking. In: ICCV, pp. 1323–1330 (2011)

    Google Scholar 

  2. Wang, J., Xu, Y., Shum, H.Y., Cohen, M.F.: Video tooning. In: SIGGRAPH, pp. 572–581 (2004)

    Google Scholar 

  3. van den Hengel, A., Dick, A., Thormählen, T., Ward, B., Torr, P.H.: Videotrace: rapid interactive scene modelling from video. In: SIGGRAPH, vol. 26 (2007)

    Google Scholar 

  4. Reso, M., Jachalsky, J., Rosenhahn, B., Ostermann, J.: Temporally consistent superpixels. In: ICCV, pp. 385–392 (2013)

    Google Scholar 

  5. Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in nd images. In: ICCV, vol. 1, pp. 105–112 (2001)

    Google Scholar 

  6. Rother, C., Kolmogorov, V., Blake, A.: Grabcut: Interactive foreground extraction using iterated graph cuts. In: SIGGRAPH, vol. 23, pp. 309–314 (2004)

    Google Scholar 

  7. Delong, A., Boykov, Y.: A scalable graph-cut algorithm for nd grids. In: CVPR, pp. 1–8 (2008)

    Google Scholar 

  8. Scheuermann, B., Schlosser, M., Rosenhahn, B.: Efficient pixel-grouping based on dempster’s theory of evidence for image segmentation. In: Lee, K.M., Matsushita, Y., Rehg, J.M., Hu, Z. (eds.) ACCV 2012, Part I. LNCS, vol. 7724, pp. 745–759. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  9. Ren, X., Malik, J.: Learning a classification model for segmentation. In: ICCV, pp. 10–17 (2003)

    Google Scholar 

  10. Galasso, F., Cipolla, R., Schiele, B.: Video segmentation with superpixels. In: ACCV 2013, pp. 760–774 (2013)

    Google Scholar 

  11. Vazquez-Reina, A., Avidan, S., Pfister, H., Miller, E.: Multiple hypothesis video segmentation from superpixel flows. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 268–281. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  12. Grundmann, M., Kwatra, V., Han, M., Essa, I.: Efficient hierarchical graph-based video segmentation. In: CVPR, pp. 2141–2148 (2010)

    Google Scholar 

  13. Chang, J., Wei, D., Fisher III, J.W.: A video representation using temporal superpixels. In: CVPR, pp. 2051–2058 (2013)

    Google Scholar 

  14. Bergh, M.V.D., Roig, G., Boix, X., Manen, S., Gool, L.V.: Online video seeds for temporal window objectness. In: ICCV, pp. 377–384 (2013)

    Google Scholar 

  15. Xu, C., Whitt, S., Corso, J.J.: Flattening supervoxel hierarchies by the uniform entropy slice. In: ICCV, pp. 2240–2247 (2013)

    Google Scholar 

  16. Wang, J., Bhat, P., Colburn, R.A., Agrawala, M., Cohen, M.F.: Interactive video cutout. SIGGRAPH 24, 585–594 (2005)

    Article  Google Scholar 

  17. Price, B.L., Morse, B.S., Cohen, S.: Livecut: Learning-based interactive video segmentation by evaluation of multiple propagated cues. In: ICCV, pp. 779–786 (2009)

    Google Scholar 

  18. Bai, X., Wang, J., Simons, D., Sapiro, G.: Video snapcut: robust video object cutout using localized classifiers. SIGGRAPH 28 (2009)

    Google Scholar 

  19. Dondera, R., Morariu, V., Wang, Y., Davis, L.: Interactive video segmentation using occlusion boundaries and temporally coherent superpixels. In: WACV, pp. 784–791 (2014)

    Google Scholar 

  20. Levinshtein, A., Sminchisescu, C., Dickinson, S.: Spatiotemporal closure. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 369–382. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  21. Lermé, N., Malgouyres, F., Létocart, L.: Reducing graphs in graph cut segmentation. In: ICIP, pp. 3045–3048 (2010)

    Google Scholar 

  22. Scheuermann, B., Rosenhahn, B.: SlimCuts: GraphCuts for high resolution images using graph reduction. In: Boykov, Y., Kahl, F., Lempitsky, V., Schmidt, F.R. (eds.) EMMCVPR 2011. LNCS, vol. 6819, pp. 219–232. Springer, Heidelberg (2011)

    Google Scholar 

  23. Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., Susstrunk, S.: Slic superpixels compared to state-of-the-art superpixel methods. TPAMI 34, 2274–2282 (2012)

    Article  Google Scholar 

  24. Reso, M., Jachalsky, J., Rosenhahn, B., Ostermann, J.: Superpixels for video content using a contour-based em optimization. In: ACCV (2014)

    Google Scholar 

  25. Boykov, Y., Kolmogorov, V.: An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision. TPAMI 26, 1124–1137 (2004)

    Article  Google Scholar 

  26. Sundberg, P., Brox, T., Maire, M., Arbelaez, P., Malik, J.: Occlusion boundary detection and figure/ground assignment from optical flow. In: CVPR (2011)

    Google Scholar 

  27. Galasso, F., Nagaraja, N.S., Cárdenas, T.J., Brox, T., Schiele, B.: A unified video segmentation benchmark: Annotation, metrics and analysis. In: ICCV, pp. 3527–3534 (2013)

    Google Scholar 

  28. Comaniciu, D., Meer, P.: Mean shift: A robust approach toward feature space analysis. TPAMI 24, 603–619 (2002)

    Article  Google Scholar 

  29. Blake, A., Rother, C., Brown, M., Perez, P., Torr, P.: Interactive image segmentation using an adaptive GMMRF model. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 428–441. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Leibniz Universität Hannover, Germany

    Matthias Reso, Björn Scheuermann, Bodo Rosenhahn & Jörn Ostermann

  2. Technicolor Research & Innovation Hannover, Germany

    Jörn Jachalsky

Authors
  1. Matthias Reso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Björn Scheuermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jörn Jachalsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bodo Rosenhahn
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jörn Ostermann
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University of Nevada at Reno, USA

    George Bebis

  2. NASA Ames Research Center, Moffett Field, CA, USA

    Richard Boyle

  3. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Bahram Parvin

  4. Desert Research Institute, Reno, NV, USA

    Darko Koracin

  5. The University of Texas at Dallas, 75080, Richardson, TX, USA

    Ryan McMahan

  6. NextGen Interactions, 27604, Raleigh, NC, USA

    Jason Jerald

  7. Indiana University, 46202, Indianapolis, IN, USA

    Hui Zhang

  8. Microsoft Research, 1 Microsoft Way, 98052, Redmond, WA, USA

    Steven M. Drucker

  9. University of Delaware, 19716-2712, Newark, DE, USA

    Chandra Kambhamettu

  10. Intel Corp., 95054, Santa Clara, CA, USA

    Maha El Choubassi

  11. Computer Graphics and Interactive Media Lab, Department of Computer Science, University of Houston, 77004, Houston, TX, USA

    Zhigang Deng

  12. NVIDIA, 34788, Leesburg, FL, USA

    Mark Carlson

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Reso, M., Scheuermann, B., Jachalsky, J., Rosenhahn, B., Ostermann, J. (2014). Interactive Segmentation of High-Resolution Video Content Using Temporally Coherent Superpixels and Graph Cut. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2014. Lecture Notes in Computer Science, vol 8887. Springer, Cham. https://doi.org/10.1007/978-3-319-14249-4_27

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14249-4_27

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14248-7

  • Online ISBN: 978-3-319-14249-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation