Skip to main content
SpringerLink
Log in
Book cover

Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015 pp 125–134Cite as

Using a Genetic Algorithm for Selection of Starting Conditions for the EM Algorithm for Gaussian Mixture Models

  • Conference paper
  • First Online:

  • 1007 Accesses

Part of the book series: Advances in Intelligent Systems and Computing ((AISC,volume 403))

Abstract

This paper addresses the problem of initialization of the expectation-maximization (EM) algorithm for maximum likelihood estimation of Gaussian mixture models. In order to avoid local maxima of the likelihood function, a genetic algorithm (GA) which searches for best initial conditions of the EM algorithm is proposed. In the GA, a chromosome represents a set of initial conditions, in which initial mean vectors of mixture components are feature vectors chosen from the training set. The chromosome also encodes variances of initial spherical covariance matrices of mixture components. To evaluate each chromosome in the GA we run the EM algorithm until convergence and use the obtained log likelihood as the fitness. In computational experiments our approach was applied to clustering problem and tested on two datasets from the image processing domain. The results indicate that our method outperforms the standard multiple restart EM algorithm and is at least comparable to the state-of-the art random swap EM method.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    http://vismod.media.mit.edu/vismod/imagery/VisionTexture/vistex.html.

References

  1. Andrews, J.L., McNicholas, P.D.: Using evolutionary algorithms for model-based clustering. Pattern Recognit. Lett. 34(9), 987–992 (2013)

    Article  Google Scholar 

  2. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  3. Caglar, A., Aksoy, S., Arikan, O.: Maximum likelihood estimation of Gaussian mixture models using stochastic search. Pattern Recognit. 45(7), 2804–2816 (2012)

    Article  MATH  Google Scholar 

  4. Conover, W.J.: Practical Nonparametric Statistics. Wiley, New York (1999)

    Google Scholar 

  5. Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  6. Figueiredo, M., Jain, A.: Unsupervised learning of finite mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 381–396 (2002)

    Article  Google Scholar 

  7. Fraley, C., Raftery, A.E.: Model-based clustering, discriminant analysis, and density estimation. J. Am. Stat. Assoc. 97(458), 611–631 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  8. Hastie, T., Tibshirani, R.: Discriminant analysis by Gaussian mixtures. J. R. Stat. Soc. Ser. B 58(1), 155–176 (1996)

    MathSciNet  MATH  Google Scholar 

  9. He, X., Cai, D., Shao, Y., Bao, H., Han, J.: Laplacian regularized Gaussian mixture model for data clustering. IEEE Trans. Knowl. Data Eng. 23(9), 1406–1418 (2011)

    Article  Google Scholar 

  10. Kailath, T.: The divergence and Bhattacharyya distance measures in signal selection. IEEE Trans. Commun. Technol. 15(1), 52–60 (1967)

    Article  Google Scholar 

  11. Kwedlo, W.: Estimation of parameters of Gaussian mixture models by a hybrid method combining a self-adaptive differential evolution with the EM algorithm. Adv. Comput. Sci. Res. 14, 109–123 (2014)

    Google Scholar 

  12. Kwedlo, W.: A parallel EM algorithm for Gaussian mixture models implemented on a NUMA system using OpenMP. In: Proceedings of the 22nd Euromicro International Conference on Parallel, Distributed, and Network-Based Processing PDP 2014, pp. 292–298. IEEE CPS (2014)

    Google Scholar 

  13. Lovasz, L., Plummer, M.D.: Matching Theory. American Mathematical Society, Providence (2009)

    MATH  Google Scholar 

  14. McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions. Wiley, New York (2008)

    Book  MATH  Google Scholar 

  15. McLachlan, G., Peel, D.: Finite Mixture Models. Wiley, New York (2000)

    Book  MATH  Google Scholar 

  16. Michalewicz, Z.: Genetic Algorithms + Data Structures = Evolution Programs. Springer, Berlin (1996)

    Book  MATH  Google Scholar 

  17. Nene, S.A., Nayar, S.K., Murase, H.: Columbia object image library (COIL-20). Technical report, CUCS-005-96, Columbia University (1996)

    Google Scholar 

  18. Permuter, H., Francos, J., Jermyn, I.: A study of Gaussian mixture models of color and texture features for image classification and segmentation. Pattern Recognit. 39(4), 695–706 (2006)

    Article  MATH  Google Scholar 

  19. Pernkopf, F., Bouchaffra, D.: Genetic-based EM algorithm for learning Gaussian mixture models. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1344–1348 (2005)

    Article  Google Scholar 

  20. Redner, R.A., Walker, H.F.: Mixture densities, maximum likelihood and the EM algorithm. SIAM Rev. 26(2), 195–239 (1984)

    Article  MathSciNet  MATH  Google Scholar 

  21. Reynolds, D., Quatieri, T., Dunn, R.: Speaker verification using adapted Gaussian mixture models. Digit. Signal Process. 10(1), 19–41 (2000)

    Article  Google Scholar 

  22. Zhao, Q., Hautamäki, V., Kärkkäinen, I., Fränti, P.: Random swap EM algorithm for Gaussian mixture models. Pattern Recognit. Lett. 33(16), 2120–2126 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Bialystok University of Technology grant S/WI/2/2013.

Author information

Authors and Affiliations

  1. Faculty of Computer Science, Bialystok University of Technology, Wiejska 45A, 15-351, Bialystok, Poland

    Wojciech Kwedlo

Authors
  1. Wojciech Kwedlo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wojciech Kwedlo .

Editor information

Editors and Affiliations

  1. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Robert Burduk

  2. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Konrad Jackowski

  3. Department of Systems and Computer, Wrocław University of Technology, Wroclaw, Poland

    Marek Kurzyński

  4. Dept. of Systems and Computer Networks, Wrocław University of Technology, Wroclaw, Poland

    Michał Woźniak

  5. Department of Systems, Wrocław University of Technology, Wroclaw, Poland

    Andrzej Żołnierek

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this paper

Cite this paper

Kwedlo, W. (2016). Using a Genetic Algorithm for Selection of Starting Conditions for the EM Algorithm for Gaussian Mixture Models. In: Burduk, R., Jackowski, K., Kurzyński, M., Woźniak, M., Żołnierek, A. (eds) Proceedings of the 9th International Conference on Computer Recognition Systems CORES 2015. Advances in Intelligent Systems and Computing, vol 403. Springer, Cham. https://doi.org/10.1007/978-3-319-26227-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-26227-7_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-26225-3

  • Online ISBN: 978-3-319-26227-7

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation