Skip to main content
SpringerLink
Log in

Model-based clustering, classification, and discriminant analysis via mixtures of multivariate t-distributions

The tEIGEN family

  • Published:
Statistics and Computing Aims and scope Submit manuscript

Abstract

The last decade has seen an explosion of work on the use of mixture models for clustering. The use of the Gaussian mixture model has been common practice, with constraints sometimes imposed upon the component covariance matrices to give families of mixture models. Similar approaches have also been applied, albeit with less fecundity, to classification and discriminant analysis. In this paper, we begin with an introduction to model-based clustering and a succinct account of the state-of-the-art. We then put forth a novel family of mixture models wherein each component is modeled using a multivariate t-distribution with an eigen-decomposed covariance structure. This family, which is largely a t-analogue of the well-known MCLUST family, is known as the tEIGEN family. The efficacy of this family for clustering, classification, and discriminant analysis is illustrated with both real and simulated data. The performance of this family is compared to its Gaussian counterpart on three real data sets.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Andrews, J.L., McNicholas, P.D.: Extending mixtures of multivariate t-factor analyzers. Stat. Comput. 21(3), 361–373 (2011a)

    Article  MathSciNet  Google Scholar 

  • Andrews, J.L., McNicholas, P.D.: Mixtures of modified t-factor analyzers for model-based clustering, classification, and discriminant analysis. J. Stat. Plan. Inference 141(4), 1479–1486 (2011b)

    Article  MathSciNet  MATH  Google Scholar 

  • Andrews, J.L., McNicholas, P.D., Subedi, S.: Model-based classification via mixtures of multivariate t-distributions. Comput. Stat. Data Anal. 55(1), 520–529 (2011)

    Article  MathSciNet  Google Scholar 

  • Banfield, J.D., Raftery, A.E.: Model-based Gaussian and non-Gaussian clustering. Biometrics 49(3), 803–821 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  • Baum, L.E., Petrie, T., Soules, G., Weiss, N.: A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. Ann. Math. Stat. 41, 164–171 (1970)

    Article  MathSciNet  MATH  Google Scholar 

  • Besag, J., Green, P., Higdon, D., Mengersen, K.: Bayesian computation and stochastic systems. Stat. Sci. 10(1), 3–41 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  • Bouveyron, C., Girard, S., Schmid, C.: High-dimensional data clustering. Comput. Stat. Data Anal. 52(1), 502–519 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  • Celeux, G., Govaert, G.: Gaussian parsimonious clustering models. Pattern Recognit. 28, 781–793 (1995)

    Article  Google Scholar 

  • Dasgupta, A., Raftery, A.E.: Detecting features in spatial point processes with clutter via model-based clustering. J. Am. Stat. Assoc. 93, 294–302 (1998)

    MATH  Google Scholar 

  • Day, N.E.: Estimating the components of a mixture of normal distributions. Biometrika 56, 463–474 (1969)

    Article  MathSciNet  MATH  Google Scholar 

  • Dean, N., Murphy, T.B., Downey, G.: Using unlabelled data to update classification rules with applications in food authenticity studies. J. R. Stat. Soc., Ser. C, Appl. Stat. 55(1), 1–14 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. J. R. Stat. Soc., Ser. B, Stat. Methodol. 39(1), 1–38 (1977)

    MathSciNet  MATH  Google Scholar 

  • Edwards, A.W.F., Cavalli-Sforza, L.L.: A method for cluster analysis. Biometrics 21, 362–375 (1965)

    Article  Google Scholar 

  • Forina, M., Armanino, C., Castino, M., Ubigli, M.: Multivariate data analysis as a discriminating method of the origin of wines. Vitis 25, 189–201 (1986)

    Google Scholar 

  • Fraley, C., Raftery, A.E.: How many clusters? Which clustering methods? Answers via model-based cluster analysis. Comput. J. 41(8), 578–588 (1998)

    Article  MATH  Google Scholar 

  • Fraley, C., Raftery, A.E.: Model-based clustering, discriminant analysis, and density estimation. J. Am. Stat. Assoc. 97(458), 611–631 (2002)

    Article  MathSciNet  MATH  Google Scholar 

  • Fraley, C., Raftery, A.E.: MCLUST:version 3 for R: Normal mixture modeling and model-based clustering. Technical Report 504, Department of Statistics, University of Washington (2006, September). Minor revisions January 2007 and November 2007

  • Gordon, A.D.: Classification. Chapman and Hall, London (1981)

    MATH  Google Scholar 

  • Greselin, F., Ingrassia, S.: Constrained monotone EM algorithms for mixtures of multivariate t distributions. Stat. Comput. 20(1), 9–22 (2010a)

    Article  MathSciNet  Google Scholar 

  • Greselin, F., Ingrassia, S.: Weakly homoscedastic constraints for mixtures of t-distributions. In: Fink, A., Lausen, B., Seidel, W., Ultsch, A. (eds.) Advances in Data Analysis, Data Handling and Business Intelligence. Studies in Classification, Data Analysis, and Knowledge Organization, pp. 219–228. Springer, Berlin/Heidelberg (2010b)

    Google Scholar 

  • Hastie, T., Tibshirani, R.: Discriminant analysis by Gaussian mixtures. J. R. Stat. Soc., Ser. B, Stat. Methodol. 58, 155–176 (1996)

    MathSciNet  MATH  Google Scholar 

  • Hubert, L., Arabie, P.: Comparing partitions. J. Classif. 2, 193–218 (1985)

    Article  Google Scholar 

  • Hurley, C.: Clustering visualizations of multivariate data. J. Comput. Graph. Stat. 13(4), 788–806 (2004)

    Article  MathSciNet  Google Scholar 

  • Kass, R.E., Raftery, A.E.: Bayes factors. J. Am. Stat. Assoc. 90, 773–795 (1995)

    MATH  Google Scholar 

  • Kass, R.E., Wasserman, L.: A reference Bayesian test for nested hypotheses and its relationship to the Schwarz criterion. J. Am. Stat. Assoc. 90(431), 928–934 (1995)

    MathSciNet  MATH  Google Scholar 

  • Keribin, C.: Consistent estimation of the order of mixture models Sankhyā. Indian J. Stat., Ser. A 62(1), 49–66 (2000)

    MathSciNet  MATH  Google Scholar 

  • Leroux, B.G.: Consistent estimation of a mixing distribution. Ann. Stat. 20, 1350–1360 (1992)

    Article  MathSciNet  MATH  Google Scholar 

  • Lindsay, B.G.: Mixture models: theory, geometry and applications. In: NSF-CBMS Regional Conference Series in Probability and Statistics, vol. 5. Institute of Mathematical Statistics, Hayward (1995)

    Google Scholar 

  • Mangasarian, O.L., Street, W.N., Wolberg, W.H.: Breast cancer diagnosis and prognosis via linear programming. Operations Research 43(4), 570–577 (1995)

    Article  MathSciNet  MATH  Google Scholar 

  • Maugis, C., Celeux, G., Martin-Magniette, M.-L.: Variable selection for clustering with Gaussian mixture models. Biometrics 65(3), 701–709 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  • McLachlan, G.J.: The Classification and Mixture Maximum Likelihood Approaches to Cluster Analysis. Handbook of Statistics, vol. 2, pp. 199–208. North-Holland, Amsterdam (1982)

    Google Scholar 

  • McLachlan, G.J.: Discriminant Analysis and Statistical Pattern Recognition. Wiley, New Jersey (1992)

    Book  Google Scholar 

  • McLachlan, G.J., Basford, K.E.: Mixture Models: Inference and applications to clustering. Marcel Dekker, New York (1988)

    MATH  Google Scholar 

  • McLachlan, G.J., Peel, D.: Robust cluster analysis via mixtures of multivariate t-distributions. In: Lecture Notes in Computer Science, vol. 1451, pp. 658–666. Springer, Berlin (1998)

    Google Scholar 

  • McLachlan, G.J., Peel, D.: Mixtures of factor analyzers. In: Proceedings of the Seventh International Conference on Machine Learning, pp. 599–606. Morgan Kaufmann, San Francisco (2000)

    Google Scholar 

  • McLachlan, G.J., Bean, R.W., Jones, L.B.-T.: Extension of the mixture of factor analyzers model to incorporate the multivariate t-distribution. Comput. Stat. Data Anal. 51(11), 5327–5338 (2007)

    Article  MATH  Google Scholar 

  • McNicholas, P.D.: Model-based classification using latent Gaussian mixture models. J. Stat. Plan. Inference 140(5), 1175–1181 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • McNicholas, P.D., Murphy, T.B.: Parsimonious Gaussian mixture models. Stat. Comput. 18, 285–296 (2008)

    Article  MathSciNet  Google Scholar 

  • McNicholas, P.D., Murphy, T.B.: Model-based clustering of longitudinal data. Can. J. Stat. 38(1), 153–168 (2010a)

    MathSciNet  MATH  Google Scholar 

  • McNicholas, P.D., Murphy, T.B.: Model-based clustering of microarray expression data via latent Gaussian mixture models. Bioinformatics 26(21), 2705–2712 (2010b)

    Article  Google Scholar 

  • McNicholas, P.D., Murphy, T.B., McDaid, A.F., Frost, D.: Serial and parallel implementations of model-based clustering via parsimonious Gaussian mixture models. Comput. Stat. Data Anal. 54(3), 711–723 (2010)

    Article  MathSciNet  MATH  Google Scholar 

  • Meng, X.-L., Rubin, D.B.: Maximum likelihood estimation via the ECM algorithm: a general framework. Biometrika 80, 267–278 (1993)

    Article  MathSciNet  MATH  Google Scholar 

  • Orchard, T., Woodbury, M.A.: A missing information principle: theory and applications. In: Le Cam, L.M., Neyman, J., Scott, E.L. (eds.) Proceedings of the Sixth Berkeley Symposium on Mathematical Statistics and Probability. Theory of Statistics, vol. 1, pp. 697–715. University of California Press, Berkeley (1972)

    Google Scholar 

  • R Development Core Team: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna (2010)

    Google Scholar 

  • Raftery, A.E., Dean, N.: Variable selection for model-based clustering. J. Am. Stat. Assoc. 101(473), 168–178 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  • Rand, W.M.: Objective criteria for the evaluation of clustering methods. J. Am. Stat. Assoc. 66, 846–850 (1971)

    Google Scholar 

  • Schwarz, G.: Estimating the dimension of a model. Ann. Stat. 6(2), 461–464 (1978)

    Article  MATH  Google Scholar 

  • Scrucca, L.: Dimension reduction for model-based clustering. Stat. Comput. 20(4), 471–484 (2010)

    Article  MathSciNet  Google Scholar 

  • Sundberg, R.: Maximum likelihood theory for incomplete data from an exponential family. Scand. J. Stat. 1, 49–58 (1974)

    MathSciNet  MATH  Google Scholar 

  • Titterington, D.M., Smith, A.F.M., Makov, U.E.: Statistical Analysis of Finite Mixture Distributions. Wiley, Chichester (1985)

    MATH  Google Scholar 

  • Venables, W.N., Ripley, B.D.: Modern Applied Statistics with S-PLUS. Springer, Berlin (1999)

    MATH  Google Scholar 

  • Wolfe, J.H.: A computer program for the maximum-likelihood analysis of types. USNPRA Technical Bulletin 65-15, US Naval Personal Research Activity, San Diego (1965)

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics and Statistics, University of Guelph, Guelph, Ontario, N1G 2W1, Canada

    Jeffrey L. Andrews & Paul D. McNicholas

Authors
  1. Jeffrey L. Andrews
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul D. McNicholas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paul D. McNicholas.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Andrews, J.L., McNicholas, P.D. Model-based clustering, classification, and discriminant analysis via mixtures of multivariate t-distributions. Stat Comput 22, 1021–1029 (2012). https://doi.org/10.1007/s11222-011-9272-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11222-011-9272-x

Keywords

Search

Navigation