Skip to main content
SpringerLink
Log in

Fuzzy clustering with learnable cluster-dependent kernels

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

We propose a new relational clustering approach, called Fuzzy clustering with Learnable Cluster-dependent Kernels (FLeCK), that learns the underlying cluster-dependent dissimilarity measure while seeking compact clusters. The learned dissimilarity is based on a Gaussian kernel function with cluster-dependent parameters. Each cluster’s parameter learned by FLeCK reflects the relative intra-cluster and inter-cluster characteristics. These parameters are learned by optimizing both the intra-cluster and the inter-cluster distances. This optimization is achieved iteratively by dynamically updating the partition and the local kernel. This makes the kernel learning task takes advantages of the available unlabeled data and reciprocally, the categorization task takes advantages of the learned local kernels. Another key advantage of FLeCK is that it is formulated to work on relational data. This makes it applicable to data where objects cannot be represented by vectors or when clusters of similar objects cannot be represented efficiently by a single prototype. Using synthetic and real data sets, we show that FLeCK learns meaningful parameters and outperforms several other algorithms. In particular, we show that when data include clusters with various inter- and intra-cluster distances, learning cluster-dependent parameters is crucial in obtaining a good partition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Shi J, Malik J (2000) Normalized cuts and image segmentation. In: Proceedings of IEEE transactions on pattern analysis and machine intelligence, vol 22, no 8

  2. Yu J, Liu D, Tao D, Seah HS (2012) On combining multiple features for cartoon character retrieval and clip synthesis. IEEE Trans Syst Man Cybern B Cybern 42(5):1413–1427

    Article  Google Scholar 

  3. Yu J, Liu D, Tao D, Seah HS (2011) Complex object correspondence construction in two-dimensional animation. IEEE Trans Image Process 20(11):3257–3269

  4. Yu J, Tao D (2013) Modern machine learning techniques and their applications in cartoon animation research. Wiley-IEEE Press, Hoboken, New Jersey

  5. Klawonn F (2004) Fuzzy Clustering: insights and a new approach.Mathw Soft Comput 11:125–142

  6. Xu R, Wunsch DII (2005) Survey of clustering algorithms. IEEE Trans Neural Netw 16(3):645–678

    Article  Google Scholar 

  7. Bezdek JC (1981) Pattern recognition with fuzzy objective function algorithm. Plenum Press, New York

  8. Gustafson EE, Kessel WC (1979) Fuzzy Clustering with a Fuzzy Covariance Matrix. In: Proceedings of the IEEE Conference on Decision and Control, San Diego, CA, 761–766 

  9. Hathaway RJ, Huband JM, Bezdek JC (2005) A kernelized non-euclidean relational fuzzy c-means algorithm. In: Proceedings of FUZZ-IEEE, Reno, NV, pp 414–419

  10. Guan N, Tao D, Luo Z, Shawe-Taylor J (2012) MahNMF: Manhattan non-negative matrix factorization. CoRR. arXiv:1207.3438v1

  11. Guan N, Tao D, Luo Z, Yuan B (2012) Onlinenonnegative matrix factorization with robust stochasticapproximation. IEEE Trans Neural Netw Learn Syst 23(7):1087–1099

  12. Guan N, Tao D, Luo Z, Yuan B (2011) IEEE TransNeural Netw 22(8):1218–1230

  13. Qinand AK, Suganthan PN (2004) Kernel neural gas algorithms with application to cluster analysis.  In: 17th International Conference on Pattern Recognition-ICPR, pp 617–620

  14. Inokuchi R, Miyamoto S (2004) LVQ clustering and SOM using a kernel function. IEEE Conf Fuzzy Syst, Budapest, Hungary, CD-ROM Proc. pp 1–4

  15. Girolami M (2002) Mercer kernel based clustering in feature space. In: Proceedings of IEEE transactions on neural networks, 13(3):780–784

  16. Zelnik-Manor L, Perona P (2005) Self-tuning spectral clustering. In: Proceedings of NIPS, pp 1601–1608

  17. Hathaway RJ, Bezdek JC (1994) NerF c-means: non-euclidean relational fuzzy clustering. Pattern Recognit 27(3):429–437

    Article  Google Scholar 

  18. Rhee FCH, Choi KS, Choi BI (2009) Kernel approach to possibilistic c-means clustering. Int J Intell Syst 24:272–292

    Article  MATH  Google Scholar 

  19. Bchir O, Frigui H (2011) Fuzzy clustering with learnable cluster dependent kernels. In: Proceedings of IEEE international conference on fuzzy systems, pp 2521–2527

  20. Tan P-N, Steinbach M, Kumar V (2006) Introduction todata mining. Addison Wesley, Upper Saddle River

  21. Ester M, Kriegel H-P, Sander J, Xu X (1996) A density-based algorithm for discovering clusters in large spatial databases with noise. In: Proceedings of KDD-96, pp 226–231

  22. Raju G, Thomas B, Tobgay S, Kumar TS (2008) Fuzzy clustering methods in data mining: a comparative case analysis. In: Proceedings of ICACTE, pp 489–493

  23. Hathaway RJ, Davenport JW, Bezdek JC (1989) Relational duals of the c-means algorithms. Pattern Recognit 22(2):205–212

    Article  MathSciNet  MATH  Google Scholar 

  24. Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17:395–416

    Article  MathSciNet  Google Scholar 

  25. Cortes C, Vapnik V (1995) Support vector networks. Mach Learn 20(3):273–297

    MATH  Google Scholar 

  26. Chen B, Liu H, Bao Z (2008) Optimizing the data-dependent kernel under a unified kernel optimization framework. Pattern Recognit 41:2107–2119

    Article  MATH  Google Scholar 

  27. Xion H, Swamy MNS, Ahmad MO (2005) Optimizing the kernel in the empirical feature space. IEEE Neural Netw 16(2):460–474

    Article  Google Scholar 

  28. Lanckriet GRG, Cristianini N, Bartlett P, Ghaoui LE, Jordan MI (2004) Learning the kernel matrix with semidefinite programming. J Mach Learn Res 5:27–72

    MathSciNet  MATH  Google Scholar 

  29. Cristianini N, Kandola J, Elisseeff A, Taylor JS (2002) On kernel-target alignment. Adv Neural Inf Process Syst 14:367–373

    Google Scholar 

  30. Yu J, Liu D, Tao D, Seah HS (2011) Complex objectcorrespondence construction in two-dimensional animation. IEEETrans Image Process 20(11):3257–3269

  31. Donoho DL, Grimes C (2003) Hessian eigenmaps: new locallylinear embedding techniques for high-dimensional data. Natl AcadSci USA 100(10):5591–5596

    Article  MathSciNet  MATH  Google Scholar 

  32. Liu W, Tao D, Cheng J, Tang Y (2014) Multiview Hessian discriminative sparse coding for image annotation. Comput Vis Image Underst 118:50–60

    Article  Google Scholar 

  33. Belkin M, Niyogi P, Sindhwani V, Regularization M (2006) A geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MathSciNet  MATH  Google Scholar 

  34. Weifeng L, Dacheng T (2013) Multiview Hessian regularization for image annotation. IEEE Trans Image Process 22(7):2676–2687

  35. Perona P, Freeman WT (1998) A factorization approach to grouping. In: Burkhardt H, Neumann B (eds) Computer Vision — ECCV'98. Lecture Notes in Computer Science, vol 1406. Springer, Heidelberg,  pp 655–670

  36. Broomhead D, Lowe D (1988) Multivariable functional interpolation andadaptive networks. In: Proceedings of ComplexSyst, vol 2

  37. Er MJ, Wu S, Lu J, Toh HL (2002) Face recognition with radial basis function (RBF) neural networks. In: Proceedings of IEEE transactions on neural networks, 13(3):697–710

  38. Borgelt C, Kruse R (2006) Finding the number of fuzzy clusters by resampling. In: Proceedings of IEEE conference on fuzzy systems, pp 250–256

  39. Frias-Martinez E, Chen SY, Liu X (2006) Survey of data mining approaches to user modeling for adaptive hypermedia. IEEE Trans Syst Man Cybern 36(6):734–749

    Article  Google Scholar 

  40. Philibert J (2005) One and a half century of diffusion: fick, Einstein, before and beyond. Diffus Fundam 4:6.1–6.19

    Google Scholar 

  41. Manjunath BS, Salembier P, Sikora T (2002) Introduction to MPEG 7: multimedia content description language. Wiley, New York

  42. Alimoglu F, Alpaydin E (1996) Methods of combining multiple classifiers based on different representations for pen-based handwriting recognition. TAINN, Istanbul

    Google Scholar 

  43. Frank A, Asuncion A (2010) UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml

  44. Frey PW, Slate DJ (1991) Letter recognition using Holland-style adaptive classifiers. Mach Learn 6:161–182

    Google Scholar 

  45. Levine E, Domany E (2001) Unsupervised estimation of cluster validity using resampling. Neural Comput 13:2573–2593

    Article  MATH  Google Scholar 

  46. Bellman RE (1961) Adaptive control processes. Princeton University Press, Princeton

    Book  MATH  Google Scholar 

  47. Beyer K, Goldstein J, Ramakrishnan R, Shaft U (1999) When is nearest neighbors meaningful? In: Proceedings of ICDT, pp 217–235

  48. Dhillon IS, Guan Y, Kulis B (2004) Kernel k-means: spectral clustering and normalized cuts. Proceedings of the 10th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 551–556

  49. Krishnapuram R, Keller J (1993) A possibilistic approach to clustering. IEEE Trans Fuzzy Syst 1(3):222–233

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Research Center of College of Computer and Information Sciences, King Saud University. The authors are grateful for this support.

Author information

Authors and Affiliations

  1. CS Department, College of Computer and Information Sciences, King Saud University, Riyadh, Saudi Arabia

    Ouiem Bchir & Mohamed Maher Ben Ismail

  2. Multimedia Research Lab, University of Louisville, Louisville, Kentucky, USA

    Hichem Frigui

Authors
  1. Ouiem Bchir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hichem Frigui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Maher Ben Ismail
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ouiem Bchir.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bchir, O., Frigui, H. & Ismail, M.M.B. Fuzzy clustering with learnable cluster-dependent kernels. Pattern Anal Applic 19, 919–937 (2016). https://doi.org/10.1007/s10044-015-0461-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-015-0461-7

Keywords

Search

Navigation