Skip to main content
SpringerLink
Log in

Entropy-type classification maximum likelihood algorithms for mixture models

  • Original Paper
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Mixtures of distributions are popularly used as probability models for analyzing grouped data. Classification maximum likelihood (CML) is an important maximum likelihood approach to clustering with mixture models. Yang et al. extended CML to fuzzy CML. Although fuzzy CML presents better results than CML, it is always affected by the fuzziness index parameter. In this paper, we consider fuzzy CML with an entropy-regularization term to create an entropy-type CML algorithm. The proposed entropy-type CML is a parameter-free algorithm for mixture models. Some numerical and real-data comparisons show that the proposed method provides better results than some existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum Press, New York

    MATH  Google Scholar 

  • Blake CL, Merz CJ (1998) UCI repository of machine learning databases, a huge collection of artificial and real-world data sets. http://archive.ics.uci.edu/ml/datasets.html

  • Bryant PG, Williamson JA (1978) Asymptotic behaviour of classification maximum likelihood estimates. Biometrika 65:273–281

    Article  MATH  Google Scholar 

  • Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm (with discussion). J R Stat Soc B 39:1–38

    MATH  MathSciNet  Google Scholar 

  • Everitt BS, Hand DJ (1981) Finite mixture distributions. Chapman and Hall, London

    MATH  Google Scholar 

  • Forgy EW (1965) Cluster analysis of multivariate data: efficiency versus interpretability of classifications. Biometrics 21:768–769

    Google Scholar 

  • Gustafson DE, Kessel WC (1979) Fuzzy clustering with a fuzzy covariance matrix. In: Proceedings of IEEE CDC, San Diego, CA, pp 761–766

  • Hoppner F, Klawonn F, Kruse R, Runkler T (1999) Fuzzy cluster analysis: methods for classification data analysis and image recognition. Wiley, New York

    Google Scholar 

  • Huang Z (1998) Extensions to the k-means algorithm for clustering large data sets with categorical values. Data Mining Knowl Discov 2:283–304

    Article  Google Scholar 

  • Huang Z, Ng MK (1999) A fuzzy k-modes algorithm for clustering categorical data. IEEE Trans Fuzzy Syst 7:446–452

    Article  Google Scholar 

  • Jaynes ET (1957) Information theory and statistical mechanics. Phys Rev 106:620–630

    Article  MathSciNet  Google Scholar 

  • Kim DW, Lee KH, Lee D (2004) Fuzzy clustering of categorical data using fuzzy centroids. Pattern Recogn Lett 25:1263–1271

    Article  Google Scholar 

  • MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of 5th Berkeley Symposium, pp 281–297

  • McLachlan GJ, Basford KE (1988) Mixture models: inference and applications to clustering. Marcel Dekker, New York

    MATH  Google Scholar 

  • Scott AJ, Symons MJ (1971) Clustering methods based on likelihood ratio criteria. Biometrics 27:387–397

    Article  Google Scholar 

  • Symons MJ (1981) Clustering criteria and multivariate normal mixtures. Biometrics 37:35–43

    Article  MATH  MathSciNet  Google Scholar 

  • Yang MS (1993a) A survey of fuzzy clustering. Math Comput Model 18:1–16

    Article  MATH  Google Scholar 

  • Yang MS (1993b) On a class of fuzzy classification maximum likelihood procedures. Fuzzy Sets Syst 57:365–375

    Article  MATH  Google Scholar 

  • Yang MS, Chiang YH, Chen CC, Lai CY (2008) A fuzzy k-partitions model for categorical data and its comparison to the GoM model. Fuzzy Sets Syst 159:390–405

    Article  MATH  MathSciNet  Google Scholar 

  • Yu J, Yang MS (2007) A generalized fuzzy clustering regularization model with optimality tests and model complexity analysis. IEEE Trans Fuzzy Syst 15:904–915

    Article  Google Scholar 

  • Zadeh LA (1965) Fuzzy sets. Inf Control 8:338–353

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the anonymous referees for their helpful comments in improving the presentation of this paper. This work was supported in part by the National Science Council of Taiwan under Grant NSC-97-2118-M-033-002-MY2.

Author information

Authors and Affiliations

  1. Department of Applied Mathematics, Chung Yuan Christian University, Chung-Li, 32023, Taiwan

    Chien-Yo Lai & Miin-Shen Yang

Authors
  1. Chien-Yo Lai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miin-Shen Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Miin-Shen Yang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Lai, CY., Yang, MS. Entropy-type classification maximum likelihood algorithms for mixture models. Soft Comput 15, 373–381 (2011). https://doi.org/10.1007/s00500-010-0560-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-010-0560-8

Keywords

Search

Navigation