Skip to main content
SpringerLink
Log in

A mutual information based online evolving clustering approach and its applications

  • Original Paper
  • Published:
Evolving Systems Aims and scope Submit manuscript

Abstract

In this article, a new recursive evolving clustering method is proposed based on the well-known Gustafson–Kessel algorithm. The novelty of the proposed method involves the adaptation and integration of the mutual information based formulation to accommodate the Mahalanobis distance, which functions as the similarity measure and the unification of the clustering generation and pruning mechanisms. Example applications of the method are also discussed in the areas of data compression and knowledge extraction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Abdullah O, Abdel-Qader I, Bazuin B (2016) K-means-Jensen–Shannon divergence for a WLAN indoor positioning system. In: Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), IEEE Annual (pp. 1–5). IEEE

  • Ahmed HA, Mahanta P, Bhattacharyya DK, Kalita JK (2011) Gerc: tree based clustering for gene expression data. In: Bioinformatics and bioengineering (BIBE). In Bioinformatics and bioengineering (BIBE), 2011 IEEE 11th international conference on, pp. 299–302.

  • Angelov P (2013) Evolving rule-based models: a tool for design of flexible adaptive systems (vol. 92). Physica

  • Angelov P, Filev DP, Kasabov N (eds) (2010) Evolving intelligent systems: methodology and applications (vol. 12). Wiley, New York

  • Banerjee A, Guo X, Wang H (2005a) On the optimality of conditional expectation as a Bregman predictor. IEEE Trans Inf Theory 51(7):2664–2669

    Article  MathSciNet  MATH  Google Scholar 

  • Banerjee A, Merugu S, Dhilon IS, Ghosh J (2005b) Clustering with Bregman divergences. J Mach Learn Res 6:1705–1749

    MathSciNet  MATH  Google Scholar 

  • Bezdek JC, Ehrlich R, Full W (1984) FCM: The fuzzy c-means clustering algorithm. Comput Geosci 10(23):191–203

    Article  Google Scholar 

  • Boubacar HA, Lecoeuche S (2005). A new kernel-based algorithm for online clustering. In: International conference on artificial neural networks, pp. 583–588. Springer Berlin Heidelberg

  • Bouguila N, Ziou D (2006) Online clustering via finite mixtures of Dirichlet and minimum message length. Eng Appl Artif Intell 19(4):371–379

    Article  Google Scholar 

  • Bregman LM (1967) The relaxation method of finding the common point of convex sets and its application to the solution of problems in convex programming. USSR Comput Math Math Phys 7(3):200–217

    Article  MathSciNet  MATH  Google Scholar 

  • Davies DL, Bouldin DW (1979) A cluster separation measure. IEEE Trans Pattern Anal Mach Intell 2:224–227

    Article  Google Scholar 

  • Dovžan D, Škrjanc I (2011) Recursive clustering based on a Gustafson–Kessel algorithm. Evolving Syst 2(1):15–24

    Article  Google Scholar 

  • Driver HE, Kroeber AL (1932) Quantitative expression of cultural relationships. University of California Press, Berkeley

    Google Scholar 

  • Duchi J (2007) Derivations for linear algebra and optimization. Berkeley, California. https://scholar.google.com/scholar?q=Derivations+for+linear+algebra+and+optimization&btnG=&hl=en&as_sdt=0%2C23

  • Duda RO, Hart PE, Stork DG (1973) Pattern classification. Wiley, New York, pp 526–528

    MATH  Google Scholar 

  • Filev DP, Georgieva O (2010) An extended version of the Gustafson–Kessel algorithm for evolving data stream clustering. Evolving intelligent systems: methodology and applications. IEEE Press Series on Computational Intelligence. Wiley, New York, pp 273–300

  • Filev D, Tseng F (2006a) Novelty detection based machine health prognostics. In: 2006 International symposium on evolving fuzzy systems, IEEE, pp 193–199

  • Filev D, Tseng F (2006b) Real time novelty detection modeling for machine health prognostics. In: NAFIPS 2006–2006 annual meeting of the North American fuzzy information processing society

  • Filev D, Tseng FF, Kristinsson J, McGee R (2011) Contextual on-board learning and prediction of vehicle destinations. IEEE Computational Intelligence in Vehicles and Transportation Systems (CIVTS), pp 87–91

  • Georgieva O, Filev DP (2009). Gustafson–kessel algorithm for evolving data stream clustering. In: Proceedings of the international conference on computer systems and technologies and workshop for PhD students in computing p 62, ACM

  • Gustafson DE, Kessel WC (1978) Fuzzy clustering with a fuzzy covariance matrix. Scientific Systems. Inc., Cambridge

    Book  MATH  Google Scholar 

  • Hai Y, Tseng F, Wang Q (2015) Patent No. 9121722 US

  • Hartigan JA, Wong MA (1979) Algorithm AS 136: a k-means clustering algorithm. J R Stat Soc Series C (Applied Statistics) 28(1):100–108

    MATH  Google Scholar 

  • Iglesias JA, Angelov P, Ledezma A, Sanchis A (2012) Creating evolving user behavior profiles automatically. IEEE Trans Knowl Data Eng 24(5):854–867

    Article  Google Scholar 

  • Jain AK, Murty NM, Flynn PJ (1999) Data clustering: a review. ACM Comput surv (CSUR) 31(3):264–323

    Article  Google Scholar 

  • Kasabov NK, Song Q (2002) DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction. IEEE Trans Fuzzy Syst 10(2):144–154

    Article  Google Scholar 

  • Kohonen T (2012) Self-organization and associative memory. Springer, New York

    MATH  Google Scholar 

  • Kuo CFJ, Jian BL, Wu HC, Peng KC (2012) Automatic machine embroidery image color analysis system. Part I: using Gustafson–Kessel clustering algorithm in embroidery fabric color separation. Textile Res J 82(6):571–583

    Article  Google Scholar 

  • Lin J (1991) Divergence measures based on the Shannon entropy. IEEE Trans Inf Theory 37(1):145–151

    Article  MathSciNet  MATH  Google Scholar 

  • Lughofer ED (2008) FLEXFIS: a robust incremental learning approach for evolving Takagi–Sugeno fuzzy models. IEEE Trans Fuzzy Syst 16(6):1394–1410

    Article  Google Scholar 

  • Mahalanobis PC (1936) On the generalized distance in statistics. In: Proceedings of the National Institute of Sciences (Calcutta) vol 2, pp 49–55

  • Masson MH, Denoeux T (2008) An evidential version of the fuzzy c-means algorithm. Pattern Recognit 41(4):1384–1397

    Article  MATH  Google Scholar 

  • McGee RA, Filev DP, Kristinsson JG, Tseng F, Philips AM (2012) Patent No. 13/714919 US

  • Mousselly-Sergieh H, Döller M, Egyed-Zsigmond E, Gianini G, Kosch H, Pinon JM (2014) Tag relatedness using laplacian score feature selection and adapted Jensen–Shannon divergence. In: International Conference on multimedia modeling pp 159–171. Springer International Publishing

  • Qiao J, Wang H (2008) A self-organizing fuzzy neural network and its applications to function approximation and forecast modeling. Neurocomputing 71(4):564–569

    Article  Google Scholar 

  • Serir L, Ramasso E, Zerhouni N (2012) Evidential evolving Gustafson–Kessel algorithm for online data streams partitioning using belief function theory. Int J Approx Reason 53(5):747–768

    Article  MathSciNet  Google Scholar 

  • Shankar AS, Asokan A, Sivakumar D (2016) Brain tumor classification using Gustafson–Kessel (GK) fuzzy clustering algorithm. Brain 1(05):68–72

    Google Scholar 

  • Tefaj E, Kasneci G, Rosenstiel W, Bogdan M (2012) Bayesian online clustering of eye movement data. In: Proceedings of the symposium on eye tracking research and applications, pp. 285–288. ACM

  • Tseng F, Filev DP, Makki IH, Michelini JO (2016) Patent No. 20160109243 US

  • Tyron RC (1939) Cluster analysis: correlation profile and orthometric (factor) analysis for the isolation of unities in mind and personality. Edwards brother, Incorporated, lithoprinters and publishers

  • Wagstaff K, Cardie C, Rogers S, Schrödl S (2001) Constrained k-means clustering with background knowledge. In ICML 1:577–584

    Google Scholar 

  • Wood AR, Esko T, Yang J, Vedantam S, Pers TH, Gustafsson S et al (2014) Defining the role of common variation in the genomic and biological architecture of adult human height. Nat Genet 46(11):1173–1186

    Article  Google Scholar 

  • Woodbury MA (1950) Inverting modified matrices, memorandum Rept. 42. Statistical Research Group, 316. Princeton University, Princeton, p 316

    Google Scholar 

  • Xu R, Wunsch D (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678

    Article  Google Scholar 

  • Yager RR, Filev DP (1994) Approximate clustering via the mountain method. IEEE Trans Syst Man Cybern 24(8):1279–1284

    Article  Google Scholar 

  • Yu J, Yang MS (2015) Analysis of parameter selection for Gustafson–Kessel fuzzy clustering using Jacobian matrix. IEEE Trans Fuzzy Syst 23(6):2329–2342

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Industrial and Systems Engineering, Wayne State University, 4815 Fourth Street, Detroit, MI, 48202, USA

    Fling Tseng & Ratna Babu Chinnam

  2. Modern Control Methods and Computational Intelligence, Research and Advanced Engineering, Ford Motor Company, 2101 Village Road, Dearborn, MI, 48121, USA

    Fling Tseng & Dimitar Filev

Authors
  1. Fling Tseng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dimitar Filev
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ratna Babu Chinnam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fling Tseng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tseng, F., Filev, D. & Chinnam, R.B. A mutual information based online evolving clustering approach and its applications. Evolving Systems 8, 179–191 (2017). https://doi.org/10.1007/s12530-017-9191-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12530-017-9191-y

Keywords

Search

Navigation