Abstract
This paper address the problem of online learning finite statistical mixtures of exponential families. A short review of the Expectation-Maximization (EM) algorithm and its online extensions is done. From these extensions and the description of the k-Maximum Likelihood Estimator (k-MLE), three online extensions are proposed for this latter. To illustrate them, we consider the case of mixtures of Wishart distributions by giving details and providing some experiments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Thus, \(Z_{i}\) is distributed according to the multinomial law \(\mathcal {M}_{K}(1,\{{w}_{j}\}_{j})\).
- 2.
The multinomial distribution is also an exponential family.
References
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc. Ser. B (Methodol.) 39(1), 1–38 (1977)
Titterington, D.M.: Recursive parameter estimation using incomplete data. J. R. Stat. Soc. Ser. B (Methodol.) 46(2), 257–267 (1984)
Cappé, O., Moulines, E.: On-line expectation-maximization algorithm for latent data models. J. R. Stat. Soc. Ser. B (Methodol.) 71(3), 593–613 (2009)
Neal, R.M., Hinton, G.E.: A view of the EM algorithm that justifies incremental, sparse, and other variants. In: Jordan, M.I. (ed.) Learning in Graphical Models, pp. 355–368. MIT Press, Cambridge (1999)
Nielsen, F.: On learning statistical mixtures maximizing the complete likelihood. In: Bayesian Inference and Maximum Entropy Methods in Science and Engineering (MaxEnt 2014), AIP Conference Proceedings Publishing, 1641, pp. 238–245 (2014)
Celeux, G., Govaert, G.: A classification EM algorithm for clustering and two stochastic versions. Comput. Stat. Data Anal. 14(3), 315–332 (1992)
Samé, A., Ambroise, C., Govaert, G.: An online classification EM algorithm based on the mixture model. Stat. Comput. 17(3), 209–218 (2007)
MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1(14) (1967)
Saint-Jean, C., Nielsen, F.: Hartigan’s method for k-MLE : mixture modeling with Wishart distributions and its application to motion retrieval. In: Nielsen, F. (ed.) Geometric Theory of Information. Signals and Communication Technology, pp. 301–330. Springer, Switzerland (2014)
Nielsen, F., Garcia, V.: Statistical exponential families: a digest with flash cards, November 2009. http://arxiv.org/abs/0911.4863
Wishart, J.: The generalised product moment distribution in samples from a normal multivariate population. Biometrika 20(1/2), 32–52 (1928)
Liu, Q., Ihler, A.T.: Distributed estimation, information loss and exponential families. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., Weinberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27, pp. 1098–1106. MIT Press, Cambridge (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Saint-Jean, C., Nielsen, F. (2015). Online k-MLE for Mixture Modeling with Exponential Families. In: Nielsen, F., Barbaresco, F. (eds) Geometric Science of Information. GSI 2015. Lecture Notes in Computer Science(), vol 9389. Springer, Cham. https://doi.org/10.1007/978-3-319-25040-3_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-25040-3_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25039-7
Online ISBN: 978-3-319-25040-3
eBook Packages: Computer ScienceComputer Science (R0)