Abstract
Expectation-Maximization (EM) algorithm is a typical method to estimate parameters of a model with hidden variables and is widely used for many applications. The EM algorithm is simple but sometimes overfits to specific examples and its likelihood diverges to infinite. To overcome the problem of overfitting, Shinozaki and Osterndorf have proposed the CV-EM algorithm in which the cross-validation technique is incorporated into the conventional EM algorithm, and have demonstrated validity of the algorithm with numerical experiments. In this article, we theoretically investigate properties of the CV-EM algorithm with an asymptotic analysis and reveal its mechanism of robustness.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bishop, C.: Pattern recognition and machine learning. Springer, Heidelberg (2006)
Ikeda, S.: Acceleration of the EM algorithm. Systems and Computers in Japan 31(2), 10–18 (2000)
Shimodaira, H.: A new criterion for selecting models from partially observed data. Lecture Notes In Statistics, p. 21. Springer, New York (1994)
Akaike, H.: A new look at the statistical model identification. IEEE Transactions on Automatic Control 19(6), 716–723 (1974)
Attias, H.: A variational Bayesian framework for graphical models. Advances in Neural Information Processing Systems 12(1-2), 209–215 (2000)
Shinozaki, T., Ostendorf, M.: Cross-validation and aggregated EM training for robust parameter estimation. Computer Speech & Language 22(2), 185–195 (2008)
Hampel, F.R., Rousseeuw, P.J., Ronchetti, E.M., Stahel, W.A.: Robust Statistics. Wiley, New York (1986)
Shinozaki, T., Kawahara, T.: GMM and HMM training by aggregated EM algorithm with increased ensemble sizes for robust parameter estimation. In: Proc. ICASSP, Citeseer, pp. 4405–4408 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Takenouchi, T., Ikeda, K. (2010). Theoretical Analysis of Cross-Validation(CV)-EM Algorithm. In: Diamantaras, K., Duch, W., Iliadis, L.S. (eds) Artificial Neural Networks – ICANN 2010. ICANN 2010. Lecture Notes in Computer Science, vol 6354. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15825-4_42
Download citation
DOI: https://doi.org/10.1007/978-3-642-15825-4_42
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15824-7
Online ISBN: 978-3-642-15825-4
eBook Packages: Computer ScienceComputer Science (R0)