Abstract
The expectation maximization algorithm has been classically used to find the maximum likelihood estimates of parameters in probabilistic models with unobserved data, for instance, mixture models. A key issue in such problems is the choice of the model complexity. The higher the number of components in the mixture, the higher will be the data likelihood, but also the higher will be the computational burden and data overfitting. In this work, we propose a clustering method based on the expectation maximization algorithm that adapts online the number of components of a finite Gaussian mixture model from multivariate data or method estimates the number of components and their means and covariances sequentially, without requiring any careful initialization. Our methodology starts from a single mixture component covering the whole data set and sequentially splits it incrementally during expectation maximization steps. The coarse to fine nature of the algorithm reduce the overall number of computations to achieve a solution, which makes the method particularly suited to image segmentation applications whenever computational time is an issue. We show the effectiveness of the method in a series of experiments and compare it with a state-of-the-art alternative technique both with synthetic data and real images, including experiments with images acquired from the iCub humanoid robot.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahrendt, P.: The multivariate Gaussian probability distribution. Technical report (2005). http://www2.imm.dtu.dk/pubdb/p.php?3312
Bueno, J.I., Kragic, D.: Integration of tracking and adaptive Gaussian mixture models for posture recognition. The 15th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN06), Hatfield, UK (2006)
Carpin, S., Lewis, M., Wang, J., Balakirsky, S., Scrapper, C.: Bridging the gap between simulation and reality in urban search and rescue”. In: Robocup 2006: Robot Soccer World Cup X (2006)
Comaniciu D., Meer P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Patt. Anal. Mach. Intell. 24, 603–619 (2002)
Comon P.: Independent component analysis: a new concept?. Signal Process. 36(3), 287–314 (1994)
Davis J.W., Morison A.W., Woods D.D.: An adaptive focus-of-attention model for video surveillance and monitoring. Mach. Vis. Appl. 18(1), 41–64 (2006)
Dempster A., Laird N., Rubin D.: Maximum likelihood estimation from incomplete data via the em algorithm. J. R. Stat. Soc. 30(B), 1–38 (1977)
Dobbe J.G.G., Streekstra G.J., Hardeman M.R., Ince C., Grimbergen C.A.: Measurement of the distribution of red blood cell deformability using an automated rheoscope. Cytometry (Clin. Cytometry) 50, 313–325 (2002)
Figueiredo, A., Jain, A.: Unsupervised learning of finite mixture models. IEEE Trans. Patt. Anal. Mach. Intell. 24(3) (2002)
Fritzke B.: A growing neural gas network learns topologies. Advances in Neural Inform ation Processing Systems 7 (NIPS’94), pp. 625–632. MIT Press, Cambridge MA (1995)
Greenspan H., Goldberger J., Eshet I.: Mixture model for face-color modeling and segmentation. Patt. Recognit. Lett. 22, 1525–1536 (2001)
Greggio N., Silvestri G., Menegatti E., Pagello E.: Simulation of small humanoid robots for soccer domain. J. Franklin Inst. Eng. Appl. Math. 346(5), 500–519 (2009)
Hartley H.: Maximum likelihood estimation from incomplete data. Biometrics 14, 174–194 (1958)
Henderson, N., King, R., Middleton, R.H.: An application of Gaussian mixtures: colour segmenting for the four legged league using hsi colour space. In: Proceedings of the 2008 Australasian Conference on Robotics & Automation, ISBN 978-0-646-50643-2. Canberra, Australia (2008)
Holmström, J.: Growing neural gas—experiments with gng, gng with utility and supervised gng (2002)
Hyvärinen, A., Karhunen, J., Oja, E.: Independent Component Analysis, ISBN 978-0-471-40540-5. Wiley, New York (2001)
Jensen, J.H., Ellis, D., Christensen, M.G., Jensen, S.H.: Evaluation distance measures between Gaussian mixture models of MFCCS. In: Proceedings of International Confernce on Music Information Retrieval ISMIR-07, pp. 107–108. Vienna, Austria (October 2007)
Jinachitra, P.: Constrained em estimates for harmonic source separation. In: Proceedings of the 2003 International Conference on Multimedia and Expo—vol. 1, pp. 617–620 (2003)
Ketchantang W., Derrode S., Martin L., Bourennane S.: Pearson-based mixture model for color object tracking. Mach. Vis. Appl. 19(5–6), 457–466 (2008)
Key J.: The em algorithm in medical imaging. Stat. Methods Med. Res. 6(55), 55–75 (1997)
Kohonen T.: Analysis of a simple self-organizing process. Biol. Cybern. 44(2), 135–140 (1982)
Kohonen T.: Self-organizing formation of topologically correct feature maps. Biol. Cybern. 43(1), 59–69 (1982)
Kouzani A.Z.: Classification of face images using local iterated function system. Mach. Vis. Appl. 19(4), 223–248 (2008)
Lanterman A.: Schwarz, wallace and rissanen: intertwining themes in theories of model order estimation. Int. Stat. Rev. 69, 185–212 (2001)
MacQueen, J.B.: Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)
McLachlan G., Peel D.: Finite Mixture Models. Wiley, New York (2000)
Montesano, L., Lopes, M., Bernardino, A., Santos-Victor, J.: Learning object affordances: from sensory motor maps to imitation. IEEE Trans. Robot. 24(1) (2008)
Pernkopf F., Bouchaffra D.: Genetic-based em algorithm for learning Gaussian mixture models. IEEE Trans. Patt. Anal. Mach. Intell. 27(8), 1344–1348 (2005)
Rissanen J.: Stochastic Complexity in Statistical Inquiry. Wold Scientific Publishing Co., USA (1989)
Sakimoto Y., Iahiguro M., Kitagawa G.: Akaike Information Criterion Statistics. KTK Scientific Publisher, Tokio (1986)
Shim H., Kwon D., Yun I., Lee S.: Robust segmentation of cerebral arterial segments by a sequential monte carlo method: particle filtering. Comput. Methods Prog. Biomed. 84(2-3), 135–145 (2006)
Ueda N., Nakano R., Ghahramani Y., Hiton G.: Smem algorithm for mixture models. Neural Comput. 12(10), 2109–2128 (2000)
Wallace C., Freeman P.: Estimation and inference by compact coding. J. R. Stat. Soc. B 49(3), 241–252 (1987)
Zhang Z., Chen C., Sun J., Chan K.: Em algorithms for Gaussian mixtures with split-and-merge operation. Patt. Recognit. 36, 1973–1983 (2003)
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Greggio, N., Bernardino, A., Laschi, C. et al. Fast estimation of Gaussian mixture models for image segmentation. Machine Vision and Applications 23, 773–789 (2012). https://doi.org/10.1007/s00138-011-0320-5
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-011-0320-5