Skip to main content
SpringerLink
Log in

On-line video multi-object segmentation based on skeleton model and occlusion detection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

In this work, we propose an approach for on-line video multi object segmentation based on skeleton model and occlusion detection. We consider the multi-object segmentation in every frame as a multi-class region merging based object segmentation. We then generate the initial object superpixels automatically using a skeleton model from the second frame. Moreover, we also propose an initial background superpixel prediction scheme. In case the occlusion to affect the final segmentation result, we propose an occlusion detection model based on optical flow. The experimental results show that our method is both robust in segmenting multi objects and efficient in execution time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Achanta R, Hemami S, Estrada F, Susstrunk S (2009) Frequency-tuned salient region detection. In: Computer vision and pattern recognition. CVPR 2009. IEEE conference on, pp 1597–1604

  2. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC Superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  3. Bai X, Wang J, Simons D, Sapiro G (2009) Video SnapCut: robust video object cutout using localized classifiers. ACM Trans Graph 28(3):1–11

    Article  Google Scholar 

  4. Bouguet J-Y (2000) Pyramidal implementation of the Lucas Kanade feature tracker: description of the algorithm. Available: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.185.585

  5. Brendel W, Todorovic S (2009) Video object segmentation by tracking regions. In: 2009 IEEE 12th international conference on computer vision, pp 833–840

  6. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  7. Dey TK, Zhao W (2002) Approximating the medial axis from the voronoi diagram with a convergence guarantee. In: Möhring R, Raman R (eds) Algorithms — ESA 2002: 10th annual European symposium Rome, Italy, September 17–21, 2002 proceedings. Springer, Berlin, pp 387–398

    Chapter  Google Scholar 

  8. Endres I, Hoiem D (2010) Category independent object proposals. In: Part V, Daniilidis K, Maragos P, Paragios N (eds) Computer vision – ECCV 2010: 11th European conference on computer vision, Heraklion, Crete, Greece, September 5–11, 2010, proceedings. Springer, Berlin, pp 575–588

    Chapter  Google Scholar 

  9. Grundmann M, Kwatra V, Han M, Essa I (2010) Efficient hierarchical graph-based video segmentation. In: 2010 IEEE conference on computer vision and pattern recognition (CVPR), pp 2141–2148

    Chapter  Google Scholar 

  10. Horn BKP, Schunck BG (1980) Determining optical flow. Massachusetts Institute of Technology, Cambridge

    Google Scholar 

  11. Jian M, Lam K-M, Dong J (2014) Facial-feature detection and localization based on a hierarchical scheme. Inf Sci 262:1–14

    Article  Google Scholar 

  12. Jian M, Lam KM, Dong J, Shen L (2015) Visual-patch-attention-aware saliency detection. IEEE Trans Cybern 45(8):1575–1586

    Article  Google Scholar 

  13. Jian M, Qi Q, Dong J, Sun X, Sun Y, Lam K-M (2017) Saliency detection using quaternionic distance based weber local descriptor and level priors. Multimed Tools Appl 1–18. https://doi.org/10.1007/s11042-017-5032-z

  14. Jian M, Qi Q, Dong J, Yin Y, Lam KM (2018) Integrating QDWD with pattern distinctness and local contrast for underwater saliency detection. J Vis Commun Image Represent 53:31–41

    Article  Google Scholar 

  15. Jian MW, Dong JY, Ma J (2011) Image retrieval using wavelet-based salient regions. Imaging Sci J 59(4):219–231

    Article  Google Scholar 

  16. Lee YJ, Kim J, Grauman K (2011) Key-segments for video object segmentation. In: 2011 International conference on computer vision, pp 1995–2002

  17. Levinshtein A, Stere A, Kutulakos KN, Fleet DJ, Dickinson SJ, Siddiqi K (2009) TurboPixels: fast Superpixels using geometric flows. IEEE Trans Pattern Anal Mach Intell 31(12):2290–2297

    Article  Google Scholar 

  18. Li F, Kim T, Humayun A, Tsai D, Rehg JM (2013) Video segmentation by tracking many figure-ground segments. In: 2013 IEEE international conference on computer vision, pp 2192–2199

    Chapter  Google Scholar 

  19. Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recogn 43(2):445–456

    Article  Google Scholar 

  20. Ochs P, Malik J, Brox T (2014) Segmentation of moving objects by long term video analysis. IEEE Trans Pattern Anal Mach Intell 36(6):1187–1200

    Article  Google Scholar 

  21. Papazoglou A, Ferrari V (2013) Fast object segmentation in unconstrained video. In: 2013 IEEE international conference on computer vision, pp 1777–1784

  22. Taylor B, Karasev V, Soattoc S (2015) Causal video object segmentation from persistence of occlusions. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 4268–4276

    Chapter  Google Scholar 

  23. Varas D, Marques F (2014) Region-based particle filter for video object segmentation. In: 2014 IEEE conference on computer vision and pattern recognition, pp 3470–3477

  24. Wenguan W, Jianbing S, Porikli F (2015) Saliency-aware geodesic video object segmentation. In: 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp 3395–3402

  25. Willert V, Eggert J, Clever S, Körner E (2005) Probabilistic color optical flow. In: Kropatsch WG, Sablatnig R, Hanbury A (eds) Pattern recognition: 27th DAGM symposium, Vienna, Austria, august 31–September 2, 2005. Proceedings. Springer, Berlin, pp 9–16

    Chapter  Google Scholar 

  26. Zhang X, Shan Y, Wei W, Zhu Z (2010) An image segmentation method based on improved watershed algorithm. In: Computational and information sciences (ICCIS), 2010 international conference on, pp 258–261

  27. Zhang D, Javed O, Shah M (2013) Video object segmentation through spatially accurate and temporally dense extraction of primary object regions. In: 2013 IEEE conference on computer vision and pattern recognition (CVPR), pp 628–635

    Chapter  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the Research Committee of the University of Macau under Grants MYRG2015-00011-FST and MYRG2015-00012-FST, the Science and Technology Development Fund of Macau SAR under Grants 093/2014/A2 and 041/2017/A1, and the project (2018 - 2020, Video Multi-object Co-segmentation Based on Superpixel, National Natural Science Foundation of China (NSFC) Grant No. 61702111).

Author information

Authors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Guoheng Huang

  2. University of Macau, Macau, SAR, China

    Chi-Man Pun

Authors
  1. Guoheng Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chi-Man Pun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chi-Man Pun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Huang, G., Pun, CM. On-line video multi-object segmentation based on skeleton model and occlusion detection. Multimed Tools Appl 77, 31313–31329 (2018). https://doi.org/10.1007/s11042-018-6208-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-6208-x

Keywords

Search

Navigation