Abstract
A finite mixture model using the multivariate t distribution has been well recognized as a robust extension of Gaussian mixtures. This paper presents an efficient PX-EM algorithm for supervised learning of multivariate t mixture models in the presence of missing values. To simplify the development of new theoretic results and facilitate the implementation of the PX-EM algorithm, two auxiliary indicator matrices are incorporated into the model and shown to be effective. The proposed methodology is a flexible mixture analyzer that allows practitioners to handle real-world multivariate data sets with complex missing patterns in a more efficient manner. The performance of computational aspects is investigated through a simulation study and the procedure is also applied to the analysis of real data with varying proportions of synthetic missing values.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Basford KE, McLachlan GJ (1985) Estimation of allocation rates in a cluster analysis context. J Am Stat Assoc 80: 286–293
Bensmail H, Celeux G, Raftery AE, Robert CP (1997) Inference in model-based cluster analysis. Stat Comput 7: 1–10
Brooks SP, Giudici P, Roberts GO (2003) Efficient construction of reversible jump Markov chain Monte Carlo proposal distributions (with discussion). J R Stat Soc Ser B 65: 3–55
Campbell NA, Mahon RJ (1974) A multivariate study of variation in two species of rock crab of genus Leptograpsus. Aust J Zool 22: 417–425
Chib S, Greenberg E (1995) Understanding the Metropolis–Hastings algorithm. Am Stat 49: 327–335
Dellaportas P, Papageorgiou I (2006) Multivariate mixtures of normals with unknown number of components. Stat Comput 16: 57–68
Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J R Stat Soc Ser B 39: 1–38
Diebolt J, Robert CP (1994) Estimation of finite mixture distributions through Bayesian sampling. J R Stat Soc Ser B 56: 363–375
Escobar MD, West M (1995) Bayesian density estimation and inference using mixtures. J Am Stat Assoc 90: 577–588
Frühwirth-Schnatter S (2006) Finite mixture and Markov switching models. Springer, New York
Ghahramani Z, Jordan MI (1994) Supervised learning from incomplete data via an EM approach. In: Cowan JD, Tesarro G, Alspector J (eds) Advances in neural information processing systems, vol 6. Morgan Kaufmann, San Francisco, pp 120–127
Green PJ (1995) Reversible jump Markov chain Monte Carlo computation and Bayesian model determination. Biometrika 82: 711–732
Hastings WK (1970) Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57: 97–109
Lin TI, Lee JC, Ho HJ (2006) On fast supervised learning for normal mixture models with missing information. Pattern Recogn 39: 1177–1187
Lin TI, Lee JC, Ni HF (2004) Bayesian analysis of mixture modelling using the multivariate t distribution. Stat Comput 14: 119–130
Little RJA, Rubin DB (2002) Statistical analysis with missing data, 2nd edn. Wiley, New York
Liu CH, Rubin DB (1994) The ECME algorithm: a simple extension of EM and ECM with faster monotone convergence. Biometrika 81: 633–648
Liu CH, Rubin DB (1995) ML estimation of the t distribution using EM and its extensions, ECM and ECME. Stat Sin 5: 19–39
Liu CH, Rubin DB, Wu YN (1998) Parameter expansion to accelerate EM: the PX-EM algorithm. Biometrika 85: 755–770
McLachlan GJ, Basford KE (1988) Mixture models: inference and application to clustering. Marcel Dekker, New York
McLachlan GJ, Peel D (2000) Finite mixture models. Wiely, New York
Meng XL, Rubin DB (1993) Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80: 267–278
Meng XL, van Dyk D (1997) The EM algorithm—an old folk song sung to a fast new tune (with discussion). J R Stat Soc Ser B 59: 511–567
Peel D, McLachlan GJ (2000) Robust mixture modeling using the t distribution. Stat Comput 10: 339–348
Richardson S, Green PJ (1997) On Bayesian analysis of mixtures with an unknown number of components. J R Stat Soc Ser B 59: 731–792
Rubin DB (1976) Inference and missing data. Biometrika 63: 581–592
Schafer JL (1997) Analysis of incomplete multivariate data. Chapman and Hall, London
Shoham S (2002) Robust clustering by deterministic agglomeration EM of mixtures of multivariate t-distributions. Pattern Recogn 35: 1127–1142
Shoham S, Fellows MR, Normann RA (2003) Robust, automatic spike sorting using mixtures of multivariate t-distributions. J Neurosci Methods 127: 111–122
Stone M (1974) Cross-validatory choice and assessment of statistical prediction (with discussion). J R Stat Soc Ser B 36: 111–147
Titterington DM, Smith AFM, Markov UE (1985) Statistical analysis of finite mixture distributions. Wiely, New York
Wang HX, Zhang QB, Luo B, Wei S (2004) Robust mixture modelling using multivariate t distribution with missing information. Pattern Recogn Lett 25: 701–710
Zhang ZH, Chan KL, Wu YM, Chen CB (2004) Learning a multivariate gaussian mixture model with the reversible jump MCMC algorithm. Stat Comput 14: 343–355
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Lin, TI., Ho, H.J. & Shen, P.S. Computationally efficient learning of multivariate t mixture models with missing information. Comput Stat 24, 375–392 (2009). https://doi.org/10.1007/s00180-008-0129-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00180-008-0129-5