Skip to main content
SpringerLink
Log in

On the use of divergence distance in fuzzy clustering

  • Published:
Fuzzy Optimization and Decision Making Aims and scope Submit manuscript

Abstract

Clustering algorithms divide up a dataset into a set of classes/clusters, where similar data objects are assigned to the same cluster. When the boundary between clusters is ill defined, which yields situations where the same data object belongs to more than one class, the notion of fuzzy clustering becomes relevant. In this course, each datum belongs to a given class with some membership grade, between 0 and 1. The most prominent fuzzy clustering algorithm is the fuzzy c-means introduced by Bezdek (Pattern recognition with fuzzy objective function algorithms, 1981), a fuzzification of the k-means or ISODATA algorithm. On the other hand, several research issues have been raised regarding both the objective function to be minimized and the optimization constraints, which help to identify proper cluster shape (Jain et al., ACM Computing Survey 31(3):264–323, 1999). This paper addresses the issue of clustering by evaluating the distance of fuzzy sets in a feature space. Especially, the fuzzy clustering optimization problem is reformulated when the distance is rather given in terms of divergence distance, which builds a bridge to the notion of probabilistic distance. This leads to a modified fuzzy clustering, which implicitly involves the variance–covariance of input terms. The solution of the underlying optimization problem in terms of optimal solution is determined while the existence and uniqueness of the solution are demonstrated. The performances of the algorithm are assessed through two numerical applications. The former involves clustering of Gaussian membership functions and the latter tackles the well-known Iris dataset. Comparisons with standard fuzzy c-means (FCM) are evaluated and discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abrantes A.J. and Marques J.S. (1996). A class of constrained clustering algorithms for object boundary extraction. IEEE Transactions of Image Processing 5: 1507–1521

    Article  Google Scholar 

  • Anderson E. (1935). The iris of the Gaspe peninsula. Bulletin American Society 59: 2–5

    Google Scholar 

  • Anderberg M.R. (1973). Cluster analysis for applications. Academic Press, Inc., New York

    MATH  Google Scholar 

  • Bezdek J.C. (1980). A convergence theorem for the fuzzy ISODATA clustering algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence 2: 1–8

    Article  MATH  Google Scholar 

  • Bezdek J.C. (1981). Pattern recognition with fuzzy objective function algorithms. Plenum, New York

    MATH  Google Scholar 

  • Bezdek J.C., Coray C. and Gunderson R. (1981). Detection and characterization of cluster structure. SIAM Journal of Applied Mathematics, Part 1 40(2): 339–357

    Article  MATH  MathSciNet  Google Scholar 

  • Bezdek J.C., Hathaway R.J., Howard R.E., Wilson C.A. and Windham M.P. (1987). Local convergence analysis of a grouped variable version of coordinate descent. Journal of Optimal Theory & Applications 54(3): 471–477

    Article  MATH  MathSciNet  Google Scholar 

  • Dave R.N. (1990). Fuzzy shell-clustering and applications to circle detection in digital images. Journal of General Systems 16: 343–355

    Article  MathSciNet  Google Scholar 

  • Dave R.N. and Krishnapuram R. (1997). Robustering methods: A unified view. IEEE Transactions of Fuzzy Sets and Systems 5(2): 270–293

    Article  Google Scholar 

  • Dun J.C. (1973). A Fuzzy relative of the ISODATA process and its use in detecting compact well separated clusters. Journal of Cybernetics 3(3): 32–57

    Article  MathSciNet  Google Scholar 

  • Gath I. and Geva A.B. (1989). Unsupervised optimal fuzzy clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 11(7): 773–777

    Article  Google Scholar 

  • Gowda K.C. and Krishna G. (1977). Agglomerative clustering using the concept of mutual nearest neighborhood. Pattern Recognition 10: 105–112

    Article  Google Scholar 

  • Gowda K.C. and Diday E. (1992). Symbolic clustering using a new dissimilarity measure. IEEE Transactions on Systems, Man and Cybernetics 22: 368–378

    Article  Google Scholar 

  • Gustafson, D. E., & Kessel, W. C. (1979). Fuzzy clustering with fuzzy covariance matrix. In Proceedings of IEEE CDC. San Diego, CA, pp. 761–766.

  • Ichino M. and Yaguchi H. (1994). Generalized Minkowski metrics for mixed feature-type data analysis. IEEE Transaction on Systems Man Cybernetics 24: 698–708

    Article  MathSciNet  Google Scholar 

  • Jain A.K., Murty M.N. and Flynn P.J. (1999). Data clustering: A review. ACM Computing Surveys 31(3): 264–323

    Article  Google Scholar 

  • Jarvis R.A. and Patrick E.A. (1973). Clustering using a similarity method based on shared near neighbors. IEEE Transactions on Computations C-22(8): 1025–1034

    Article  Google Scholar 

  • Krusinska E. (1987). A valuation of state of object based on weighted Mahalanobis distance. Pattern Recognition 20: 413–418

    Article  Google Scholar 

  • Kullback, S. (1968). Information theory and statistics (2nd ed.). Dover.

  • Looney C.G. (2002). Interactive clustering and merging with a new fuzzy expected value. Pattern Recognition 35: 2413–2423

    Article  MATH  Google Scholar 

  • Mao J. and Jain A.K. (1996). A self-organizing network for hyperellipsoidal clustering (HEC). IEEE Transactions on Neural Networks 7: 16–29

    Article  Google Scholar 

  • Nefti, S., & Oussalah, M. (2004). Probabilistic-fuzzy clustering algorithm. In Proceedings of IEEE International Conference on Systems, Man and Cybernetics, October, La Hague, Netherlands, Vol. 5, pp. 4786–4791.

  • Nefti, S., & Oussalah, M. (2007). Inclusion based fuzzy clustering, In J. V. de Oliveira & W. Pedrycz (Eds.), Advances in fuzzy clustering and its applications (pp. 193–208). John Wiley & Sons.

  • Nefti, S., Oussalah, M., & Rezki, Y. (2008). A modified fuzzy clustering for document retrieval. Application to document categorization. Journal of the Operational Research Society (JORS), Published online 23 Feb 2008.

  • Santini S. and Jain R. (1999). Similarity measures. IEEE Transactions on Pattern Analysis and Machine Intelligence 21(9): 871–883

    Article  Google Scholar 

  • Setnes S., Babuska R., Kaymak U. and van Nauta Lemke (1998). Similarity measures in fuzzy rule base simplification. Transactions on Systems, Man and Cybernetics, Part B 28(3): 376–386

    Article  Google Scholar 

  • Yang M.S. and Wu K.L. (2004). A Similarity-based robust clustering. IEEE Transactions on Pattern Analysis and Machine Intelligence 26(4): 434–448

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Electronics Electrical and Computer Engineering, University of Birmingham, Edgbaston, Birmingham, B15 2TT, UK

    M. Oussalah

  2. Computer Science, School of Sciences, University of Salford, Salford, M5 4WT, UK

    Samia Nefti

Authors
  1. M. Oussalah
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Samia Nefti
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Oussalah.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Oussalah, M., Nefti, S. On the use of divergence distance in fuzzy clustering. Fuzzy Optim Decis Making 7, 147–167 (2008). https://doi.org/10.1007/s10700-008-9028-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10700-008-9028-z

Keywords

Search

Navigation