Skip to main content
SpringerLink
Log in

Unsupervised learning of arbitrarily shaped clusters using ensembles of Gaussian models

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

We propose a new clustering algorithm, called SyMP, which is based on synchronization of pulse-coupled oscillators. SyMP represents each data point by an Integrate-and-Fire oscillator and uses the relative similarity between the points to model the interaction between the oscillators. SyMP is robust to noise and outliers, determines the number of clusters in an unsupervised manner, and identifies clusters of arbitrary shapes. The robustness of SyMP is an intrinsic property of the synchronization mechanism. To determine the optimum number of clusters, SyMP uses a dynamic and cluster dependent resolution parameter. To identify clusters of various shapes, SyMP models each cluster by an ensemble of Gaussian components. SyMP does not require the specification of the number of components for each cluster. This number is automatically determined using a dynamic intra-cluster resolution parameter. Clusters with simple shapes would be modeled by few components while clusters with more complex shapes would require a larger number of components. The proposed clustering approach is empirically evaluated with several synthetic data sets, and its performance is compared with GK and CURE. To illustrate the performance of SyMP on real and high-dimensional data sets, we use it to categorize two image databases.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. The CURE algorithm was provided by Dr. Han from the Department of Computer Science and Engineering, University of Minnesota.

  2. Several images in this database are 64×64 subimages extracted from a 512×512 images.

References

  1. Duda RO, Hart PE (1973) Pattern classification and scene analysis. Wiley, New York

    Google Scholar 

  2. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc B 39(1):1–38

    Google Scholar 

  3. Kaufman L, Rousseeuw PJ (1990) Finding groups in data: an introduction to cluster analysis. Addison Wesley, New York

    Google Scholar 

  4. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithms. Plenum, New York

    Google Scholar 

  5. Krishnapuram R, Nasraoui O, Frigui H (1992) The fuzzy C spherical shells algorithms: a new approach. IEEE Trans 3(5):663–671

    Google Scholar 

  6. Krishnapuram R, Frigui H, Nasraoui O (1995) Fuzzy and possibilistic shell clustering algorithms and their application to boundary detection and surface approximation I. IEEE Trans 3(1):29–43

    Google Scholar 

  7. Frigui H, Krishnapuram R (1999) A robust competitive clustering algorithm with applications in computer vision. IEEE Trans, PAMI 21(5):450–465

    Google Scholar 

  8. Jolion JM, Meer P, Bataouche S (1991) Robust clustering with applications in computer vision. IEEE Trans, PAMI 13(8):791–802

    Google Scholar 

  9. Darrell T, Pentland P (1995) Cooperative robust estimation using layers of support. IEEE Trans, PAMI 17(5):474–487

    Google Scholar 

  10. Stewart CV (1995) MINPRAN: a new robust estimator for computer vision. IEEE Trans, PAMI 17(10):925–938

    Google Scholar 

  11. Sneath PHA, Sokal RR (1973) Numerical taxonomy - the principles and practices of numerical classification. W. H. Freeman, San Francisco

    Google Scholar 

  12. Ester M, Kriegel HP, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of the 2nd international conference on knowledge discovery and data mining (KDD96) Portland Oregon

  13. Xiaowei X, Ester M, Kriegel H, Sander J (1998) A distribution-based clustering algorithm for mining in large spatial databases. In: Proceedings of the 14th international conference on data engineering, Orlando, Florida, pp 324–331

  14. Hinneburg A, Keim D (1998) An Efficient approach to clustering in large multimedia databases with noise. In: Proceedings of the 4th international conference on knowledge discovery and data mining, New York, NY, pp 58–65

  15. Ankerst M, Breunig MM, Kriegel H-P, Sander J (1999) OPTICS: ordering points to identify the clustering structure. In: Proceedings of the international conference on management of data (SIGMOD), Philadelphia PA

  16. Gustafson EE, Kessel WC (1979) Fuzzy clustering with a fuzzy covariance matrix. IEEE CDC, San Diego, California, pp 761–766

  17. Rhouma MB, Frigui H (2001) Self-organization of a population of coupled oscillators with application to clustering. IEEE Trans, PAMI 23(2)

  18. Figueiredo M, Jain A (2000) Unsupervised learning of finite mixture models. IEEE Trans, PAMI 24(3):381–396

    Google Scholar 

  19. Kuramoto Y (1984) Chemical oscillations waves and turbulence. Springer, Berlin Heidelberg New York

    Google Scholar 

  20. Strogatz SH, Mirollo RE (1991) Stability of incoherence in a population of coupled oscillators. J Stat Phys 61:613–635

    Article  Google Scholar 

  21. Winfree AT (1967) Biological rhythms and the behavior of populations of coupled oscillators. J Theor Biol 16:15–42

    Article  PubMed  Google Scholar 

  22. Buck J, Buck E (1976) Synchronous f.ireflies Scientific american, 234:74–85

    PubMed  Google Scholar 

  23. Bélair J (1988) Periodic pulsatile stimulation of a nonlinear oscillator. J Math Biol 24:74–85

    Google Scholar 

  24. Keener JP, Hoppensteadt FC, Rinzel J (1981) Integrate and fire models of nerve membrane response to oscillatory input SIAM. J Appl Math 41:734–766

    Google Scholar 

  25. Mirollo RE, Strogatz SH (1990) Synchronization of pulse coupled biological oscillators. SIAM. J Appl Math 50:1645–1662

    Article  Google Scholar 

  26. Coombes S, Lord GJ (1997) Desynchronization of pulse-soupled integrate-and-fire neurons. Phys Rev E 55:2104–2107

    Article  Google Scholar 

  27. Bressloff BC, Coombes S (1999) Symmetry and phase-locking in a ring of pulse-coupled oscillators with distributed delays. Physica D 126:99–122

    Google Scholar 

  28. Ernst U, Pawelzik K, Geisel T (1995) Synchronization induced by temporal delays in pulse-coupled oscillators. Phys Rev Lett 74:1570–1573

    Article  PubMed  Google Scholar 

  29. Nischwitz A, Glunder H (1995) Local lateral inhibition: a key to spike synchronization. Biol Cybern 73:389–400

    Google Scholar 

  30. Terman D, Wang DL (1995) Global competition and local cooperation in a network of neural oscillators. Physica D 81:148–176

    Google Scholar 

  31. Horn D, Opher I (1999) Collective excitaion phenomena and their applications. In: Maass W, Bishop CM (eds) Pulsed neural networks. MIT Press, Cambridge, pp 297–320

    Google Scholar 

  32. Gersho A, Gray RM (1992) Vector quantization and signal compression. Kluwer, Dordrecht

    Google Scholar 

  33. Kohonen T (1989) Self-organization and associative memory. Springer, Berlin Heidelberg New York

    Google Scholar 

  34. Carpenter G, Grossberg S (1996) Adaptive resonance theory (ART). In: Arbib M (ed) Handbook of Brain Theory and Neural Networks. MIT Press, Cambridge, pp 79–82

    Google Scholar 

  35. Guha S, Rastogi R, Shim K (1998) CURE: an efficient clustering algorithm for large data databases. In: Proceedings of the ACM SIGMOD conference on management of data, Seattle Washington

  36. Gath I, Geva AB (1989) Unsupervised optimal fuzzy clustering. IEEE Trans, PAMI 11(7):773–781

    Google Scholar 

  37. Bezdek JC, Keller J, Krishnapuram R, Pal NR (1999) Fuzzy models and algorithms for pattern recognition and image processing. Kluwer, Dordrecht

    Google Scholar 

  38. Theodoridis S, Koutroumbas K (1998) Pattern recognition. Academic, New York, pp 150–154

    Google Scholar 

  39. Rissanen J (1989) Stochastic complexity in statistical inquiry. World Scientific

  40. Nene SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-20) Department of Computer Science, Columbia University http://www.cs.columbia.edu/CAVE/

  41. Niemann H (1990) Pattern analysis and understanding. Springer, Berlin Heidelberg New York

    Google Scholar 

  42. Huang J, Kumar SR, Mitra M, Zu W-J (1998) Spatial color indexing and applications. In: Proceedings of ICCV ’98, Bombay

  43. Jolliffe IT (1986) Principal component analysis. Springer, Berlin Heidelberg New York

    Google Scholar 

Download references

Acknowledgements

The author would like to thank Dr. Han from the Dept. of Computer Science & Eng., Univ. of Minnesota for providing the code for the CURE algorithm and Dr. Boujemaa from the IMEDIA research group at INRIA, France for providing the image database. This material is based upon work supported by the National Science Foundation under award No. IIS-0133415.

Author information

Authors and Affiliations

  1.  Computer Engineering and Computer Science Department,  University of Louisville,  Louisville,  KY, 40292 ,  USA

    Hichem Frigui

Authors
  1. Hichem Frigui
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hichem Frigui.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Frigui, H. Unsupervised learning of arbitrarily shaped clusters using ensembles of Gaussian models. Pattern Anal Applic 8, 32–49 (2005). https://doi.org/10.1007/s10044-005-0240-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-005-0240-y

Keywords

Search

Navigation