Skip to main content
SpringerLink
Log in

Fuzzy segmentation of video shots using hybrid color spaces and motion information

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Video segmentation can be defined as the process of partitioning video into spatio-temporal objects that are homogeneous in some feature space, with the choice of features being very important to the success of the segmentation process. Fuzzy segmentation is a semi-automatic region-growing segmentation algorithm that assigns to each element in an image a grade of membership in an object. In this paper, we propose an extension of the multi-object fuzzy segmentation algorithm to segment pre-acquired color video shots. The color features are selected from channels belonging to different color models using two different heuristics: one that uses the correlations between the color channels to maximize the amount of information used in the segmentation process, and one that chooses the color channels based on the separation of the clusters formed by the seed spels for all possible color spaces. Motion information is also incorporated into the segmentation process by making use of dense optical flow maps. We performed experiments on synthetic videos, with and without noise, as well as on some real videos. The experiments show promising results, with the segmentations of real videos produced using hybrid color spaces being more accurate than the ones produced using three other color models. We also show that our method compares favorably to a state-of-the art video segmentation algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The segmentations produced by this methods were obtained using the web service provided at http://neumann.cc.gt.atl.ga.us/segmentation/.

References

  1. Moghaddam H, Lerallut J (1998) Volume visualization of the heart using MRI 4D cardiac images. J Comput Inform Tech 6:215–228

    Google Scholar 

  2. Carvalho B, Gau C, Herman G, Kong T (1999) Algorithms for fuzzy segmentation. Pattern Anal Appl 2(1):73–81

    Article  Google Scholar 

  3. Carvalho B, Garduño E, Herman G (2001) Multiseeded fuzzy segmentation on the face centered cubic grid. In: Singh S, Murshed N, Kropatsch W (eds) Advances in pattern recognition: second international conference—ICAPR 2001 (LNCS 2013). Springer, Rio de Janeiro, pp 339–348

  4. Carvalho BM, Oliveira LM, Garduno E (2006) Semi-automatic single particle segmentation on electron micrographs. In: International symposium on biomedical imaging: from nano to macro, IEEE, Washington, DC, pp 1024–1027

  5. Herman GT, Carvalho BM (2001) Multiseeded segmentation using fuzzy connectedness. IEEE Trans Pattern Analy Mach Intell 23(5):460–474

    Article  Google Scholar 

  6. Udupa J, Samarasekera S (1996) Fuzzy connectedness and object definition: theory, algorithms, and applications in image segmentation. Gr Models Image Process 58:246–261

    Article  Google Scholar 

  7. Rosenfeld A (1979) Fuzzy digital topology. Inf Control 40(1):76–87

    Article  MATH  MathSciNet  Google Scholar 

  8. Herman G (1998) Geometry of digital spaces. Birkhäuser, Boston

  9. Tekalp A (1995) Digital video processing. Prentice Hall, Upper Saddle River

  10. Bovik AC (2005) Handbook of image and video processing, 2nd edn. Elsevier Academic Press, Burlington, pp 471–489 (Ch. 4.10)

  11. Carvalho BM, Herman GT, Kong TY (2005) Simultaneous fuzzy segmentation of multiple objects. Discrete Appl Math 151(1–3):55–77

    Article  MATH  MathSciNet  Google Scholar 

  12. Carvalho BM, Oliveira LM, Andrade GS (2006) Fuzzy segmentation of color video shots. in: 13th international conference on discrete geometry for computer imagery (LNCS), vol 4245. Springer, Berlin, pp 494–500

  13. Zadeh LA (1965) Fuzzy sets. Inf Control 8:338–353

    Article  MATH  MathSciNet  Google Scholar 

  14. Johnson S (1967) Hierarchical clustering schemes. Psychometrika 32:241–254

    Article  Google Scholar 

  15. Cheng H, Jiang H, Sun Y, Wang J (2001) Color image segmentation: advances and prospects. Pattern Recognit 34(12):2259–2281

    Article  MATH  Google Scholar 

  16. Gauch J, Hsia C (1992) A comparison of three color image segmentation algorithm in four color space. In: SPIE visual communications and image processing ’92, vol 1818, pp 1168–1181

  17. Liu J, Yang YH (1994) Multiresolution color image segmentation. IEEE Trans Pattern Anal Mach Intell 16(3):689–700

    Google Scholar 

  18. VandenBroucke N, Macaire L, Posaire J (1998) Color pixels classification in an hybrid color space. In: IEEE conference on image processing, IEEE, pp 176–180

  19. Vandenbroucke N, Macaire L, Postaire JG (2003) Color image segmentation by pixel classification in an adapted hybrid color space: application to soccer image analysis. Comput Vis Image Underst 90(2):190–216

    Article  Google Scholar 

  20. Khan S, Shah M (2001) Object based segmentation of video using color, motion and spatial information. Comput Vis Pattern Recognit IEEE 2:746–751

    Google Scholar 

  21. Horn B, Schunck B (1981) Determinig optical flow. Artifi Intell 17:185–203

    Article  Google Scholar 

  22. Lucas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: 7th international joint conference on artificial intelligence, pp 674–679

  23. Singh A (1991) Optic flow computation: a unified perspective. IEEE Computer Society Press, Los Alamitos

    Google Scholar 

  24. Beauchemin SS, Barron JL (1995) The computation of optical flow. ACM Comput Surv 27(3):433–466

    Article  Google Scholar 

  25. Anandan P (1989) A computational framework and an algorithm for the measurement of visual motion. Int J Comput Vis 2:283–310

    Article  Google Scholar 

  26. McCane B, Novins K, Crannitch D, Galvin B (2001) On benchmarking optical flow. Comput Vis Image Underst 84:126–143

    Article  MATH  Google Scholar 

  27. Proesmans M, Gool LJV, Pauwels EJ, Oosterlinck A (1994) Determination of optical flow and its discontinuities using non-linear diffusion. In: ECCV ’94: Proceedings of the third European conference-volume II on computer vision, Springer, London, pp 295–304

  28. Baker S, Roth S, Scharstein D, Black M, Lewis J, Szeliski R (2007) A database and evaluation methodology for optical flow. In: International conference on computer vision, pp 1–8

  29. Galvin B, McCane B, Novins K, Mason D, Mills S (1998) Recovering motion fields: an evaluation of eight optical flow algorithms. in: Proceedings of ninth British machine vision conference, Southhampton, pp 195–204

  30. Lempitsky V, Roth S, Rother C (2008) FusionFlow: Discrete-continuous optimization for optical flow estimation, Computer Vision and Pattern Recognition. IEEE Comput Soc Conf 0:1–8

    Google Scholar 

  31. Heitz F, Bouthemy P (1993) Multimodal estimation of discontinuous optical flow using Markov random fields. IEEE Trans Pattern Anal Mach Intell 15:1217–1232

    Article  Google Scholar 

  32. Black M, Anandan P (1996) The robust estimation of multiple motions: parametric and piecewise-smooth flow fields. Comput Vis Image Underst 63:75–104

    Article  Google Scholar 

  33. Udupa J, Saha P, Lotufo R (2002) Relative fuzzy connectedness and object definition: theory, algorithms, and applications in image segmentation. IEEE Trans Pattern Anal Mach Intell 24:1485–1500

    Article  Google Scholar 

  34. Gwosdek P, Zimmer H, Grewenig S, Bruhn A, Weickert J (2010) A highly efficient GPU implementation for variational optic flow based on the Euler–Lagrange framework, Technocal Representative 267. Saarland University

  35. Gomes R, Oliveira L, Britto-Neto L, Santos T, Carvalho B, Goncalves L (2009) Producing stylized videos using the AnimVideo rendering tool. Int J Imaging Syst Technol 19:100–110

    Article  Google Scholar 

  36. Hair JF, Black B, Babin B, Anderson R, Tatham R (2005) Multivariate data analysis, 6th edn. Prentice Hall, Upper Saddle River

  37. Grundmann M, Kwatra V, Han M, Essa I (2010) Efficient hierarchical graph-based video segmentation. IEEE CVPR, pp 2141–2148

  38. Werlberger M, Trobin W, Pock T, Wedel A, Cremers D, Bischof H (2009) Anisotropic Huber-L1 optical flow. In: Proceedings of the British machine vision conference (BMVC), London, pp 1–11

  39. CVRG (2013) Optical flow algorithm evaluation. http://of-eval.sourceforge.net. Acessed Jan

  40. Kim JB, Kim HJ (2003) Efficient region-based motion segmentation for a video monitoring system. Pattern Recogn Lett 24(1–3):113–128

    Article  Google Scholar 

  41. Brito-Neto L, Carvalho B (2007) Message in a bottle: stylized rendering of sand movies. In: XX Brazilian symposium on computer graphics and image processing, pp 11–18

  42. Gomes R, Souza T, Carvalho B (2007) Mosaic animations from video inputs. In: Pacific-Rim symposium on image and video technology. LNCS 4872, pp 87–99

    Google Scholar 

  43. Carvalho B, Herman G, Kong T (2005) Simultaneous fuzzy segmentation of multiple objects. Discrete Appl Math 151:55–77

    Article  MATH  MathSciNet  Google Scholar 

  44. Dupuis A, Vasseur P (2006) Image Segmentation by Cue Selection and Integration. Image Vis Comput 10:1053–1064

    Article  Google Scholar 

  45. Grady L (2006) Random walks for image segmentation, pattern analysis and machine intelligence. IEEE Trans 28(11):1768–1783

    Google Scholar 

Download references

Acknowledgments

Bruno M. Carvalho: Supported by FAPERN/CNPq PRONEM Grant 610006/2010, CNPq Universal Grant 486951/2012-0, INCT-MACC and PROCAD grants. Edgar Garduño: This work is supported in part by the DGAPA-UNAM under Grant IN101108.

Author information

Authors and Affiliations

  1. Departamento de Informática e Matemática, Aplicada Universidade Federal do Rio Grande do Norte, Campus Universitário, SN, Lagoa Nova, Natal, RN, 59.072-970, Brazil

    Bruno M. Carvalho, Tiago S. Santos, Lucas M. Oliveira & José F. Silva Neto

  2. Departamento de Ciencias de la Computación, Instituto de Investigaciones en Matemáticas Aplicadas y en Sistemas, Universidad Nacional Autónoma de México, Cd. Universitaria, C.P. 04510, México City, México

    Edgar Garduño

Authors
  1. Bruno M. Carvalho
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Edgar Garduño
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tiago S. Santos
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lucas M. Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. José F. Silva Neto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bruno M. Carvalho.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Carvalho, B.M., Garduño, E., Santos, T.S. et al. Fuzzy segmentation of video shots using hybrid color spaces and motion information. Pattern Anal Applic 17, 249–264 (2014). https://doi.org/10.1007/s10044-013-0359-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-013-0359-1

Keywords

Search

Navigation