Skip to main content
SpringerLink
Log in

Video co-segmentation based on directed graph

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper proposes a novel video co-segmentation method, which aims to extract multi-class objects from a group of videos. A set of tracklets are first generated based on object proposals, and then a novel directed graph is constructed to connect object tracklets. The directed graph is transformed to an undirected graph, and the extraction of common object tracklets is solved by using maximum weighted clique. The obtained common object tracklets are used as seed regions to perform manifold ranking and to generate the object-level saliency maps. Based on common object tracklets and object-level saliency maps, GrabCut is exploited to get the refined co-segmentation results. Experimental results on a public video dataset show that the proposed video co-segmentation method consistently outperforms the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bron C, Kerbosch J (1973) Algorithm 457: finding all cliques of an undirected graph. Commun ACM 16(9):575–577

    Article  Google Scholar 

  2. Chen DJ, Chen HT, Chang LW (2012) Video object cosegmentation. In Proc. of ACM International Conference on Multimedia 805–808

  3. Cheng MM, Mitra NJ, Huang X, Hu SM (2014) SalientShape: group saliency in image collections. Vis Comput 30(4):443–453

    Article  Google Scholar 

  4. Chiu WC, Fritz M (2013) Multi-class video co-segmentation with a generative multi-video model. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 321–328

  5. Du H, Liu Z, Jiang J, Shen L (2013) Stretchability-aware block scaling for image retargeting. J Vis Commun Image Represent 24(4):499–508

    Article  Google Scholar 

  6. Endres I, Hoiem D (2010) Category independent object proposals. In: Proc. of European Conference on Computer Vision 575–588

  7. Faktor A, Irani M (2014) Video segmentation by non-local consensus voting. In: Proc. of British Machine Vision Conference, article 8

  8. Fu H, Xu D, Zhang B, Lin S, Ward RK (2015) Object-based multiple foreground video co-segmentation via multi-state selection graph. IEEE Trans Image Process 24(11):3415–3424

    Article  MathSciNet  Google Scholar 

  9. Guo J, Li Z, Cheong LF, Zhou SZ (2013) Video co-segmentation for meaningful action extraction. In: Proc. of IEEE International Conference on Computer Vision 2232–2239

  10. Guo J, Cheong LF, Tan RT, Zhou SZ (2014) Consistent foreground co-segmentation. In Proc. of Asian Conference on Computer Vision 241–257

  11. Huang G, Pun CM, Lin C (2017) Unsupervised video co-segmentation based on superpixel co-saliency and region merging. Multimed Tools Appl 76(10):12941–12964

    Article  Google Scholar 

  12. Jacobs D, Goldman D, Shechtman E (2010) Cosaliency: Where people look when comparing images. In: Proc. of ACM symposium on User interface software and technology 219–228

  13. Lee YJ, Kim J, Grauman K (2011) Key-segments for video object segmentation. In: Proc. of IEEE International Conference on Computer Vision 1995–2002

  14. Liu C (2009) Beyond pixels: Exploring new representations and applications for motion analysis. Ph.D. Dissertation, Massachusetts Inst. Technol., Cambridge

  15. Liu Z, Zou W, Li L, Shen L, Le Meur O (2014) Co-saliency detection based on hierarchical segmentation. IEEE Sign Proc Lett 21(1):88–92

    Article  Google Scholar 

  16. Lou Z, Gevers T (2014) Extracting primary objects by video co-segmentation. IEEE Trans Multimed 16(8):2110–2117

    Article  Google Scholar 

  17. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cyber 9(1):62–66

    Article  Google Scholar 

  18. Papazoglou A, Ferrari V (2013) Fast object segmentation in unconstrained video. In: Proc. of IEEE International Conference on Computer Vision 1777–1784

  19. Perazzi F, Wang O, Gross M, Sorkine-Hornung A (2015) Fully connected object proposals for video segmentation. In: Proc. of IEEE International Conference on Computer Vision 3227–3234

  20. Perazzi F, Pont-Tuset J, McWilliams B, Van Gool L, Gross M, Sorkine-Hornung A (2016) A benchmark dataset and evaluation methodology for video object segmentation. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 724–732

  21. Rother C, Kolmogorov V, Blake A (2004) Grabcut: interactive foreground extraction using iterated graph cuts. ACM Trans Graph 23(3):309–314

    Article  Google Scholar 

  22. Rother C, Minka T, Kolmogorov V, Blake A (2006) Cosegmentation of image pairs by histogram matching-incorporating a global constraint into mrfs. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 993–1000

  23. Rubio JC, Serrat J, López A (2012) Video co-segmentation. In Proc. of Asian Conference on Computer Vision 13–24

  24. Vicente S, Rother C, Kolmogorov V (2011) Object cosegmentation. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 2217–2224

  25. Wang W, Shen J, Li X, Porikli F (2015) Robust video object cosegmentation. IEEE Trans Image Process 24(10):3137–3148

    Article  MathSciNet  Google Scholar 

  26. Wang W, Shen J, Porikli F (2015) Saliency-aware geodesic video object segmentation. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 3395–3402

  27. Wang W, Shen J, Sun H, Shao L (2017) ViCoS2: Video co-saliency guided co-segmentation. IEEE Trans. Circuits Syst. Video Technol. doi: https://doi.org/10.1109/TCSVT.2017.2701279

  28. Wen L, Du D, Lei Z, Li SZ, Yang MH (2015) JOTS: Joint online tracking and segmentation. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 2226–2234

  29. Xiao F, Lee YJ (2016) Track and segment: An iterative unsupervised approach for video object proposals. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 933–942

  30. Y. Xie, L. Ye, Z. Liu, and X. Zhou (2016) Video co-saliency detection. In: Proc. of International Conference on Digital Image Processing, article 100335G

  31. Xu B, Bu J, Chen C, Cai D, He X, Liu W, Luo J (2011) Efficient manifold ranking for image retrieval. In: Proc. of International ACM SIGIR Conference on Research and Development in Information Retrieval 525–534

  32. Ye L, Liu Z, Li J, Zhao W, Shen L (2015) Co-saliency detection via co-salient object discovery and recovery. IEEE Sign Proc Lett 22(11):2073–2077

    Article  Google Scholar 

  33. Zhang D, Javed O, Shah M (2013) Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In: Proc. of IEEE Conference on Computer Vision Pattern Recognition 628–635

  34. Zhang D, Javed O, Shah M (2014) Video object co-segmentation by regulated maximum weight cliques. In: Proc. of European Conference on Computer Vision 551–566

  35. Zhang J, Li K, Tao W (2016) Multivideo object cosegmentation for irrelevant frames involved videos. IEEE Sign Proc Lett 23(6):785–789

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 61471230, and by the Program for Professor of Special Appointment (Eastern Scholar) at Shanghai Institutions of Higher Learning.

Author information

Authors and Affiliations

  1. Shanghai Institute for Advanced Communication and Data Science, Shanghai University, Shanghai, 200444, China

    Yufeng Xie, Zhi Liu & Xiaofei Zhou

  2. School of Communication and Information Engineering, Shanghai University, Shanghai, 200444, China

    Yufeng Xie, Zhi Liu, Xiaofei Zhou & Xuemei Zou

  3. Institute of Information and Control, Hangzhou Dianzi University, Hangzhou, 310018, China

    Xiaofei Zhou

  4. Key Laboratory of Ministry of Education for System Control and Information Processing, Shanghai Jiao Tong University, Shanghai, 200240, China

    Wei Liu

Authors
  1. Yufeng Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaofei Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xuemei Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi Liu.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xie, Y., Liu, Z., Zhou, X. et al. Video co-segmentation based on directed graph. Multimed Tools Appl 78, 10353–10372 (2019). https://doi.org/10.1007/s11042-018-6614-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6614-0

Keywords

Search

Navigation