Skip to main content
SpringerLink
Log in

A Fast Multiclass Classification Algorithm Based on Cooperative Clustering

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We present a fast multiclass classification algorithm to address the multiclass problems with a new clustering method, namely cooperative clustering. In the method of cooperative clustering, we iteratively compute the cluster centers of all classes simultaneously. For every cluster center in a class, a cluster center in an adjacent class is selected and the pair of cluster centers is drawn towards the boundary. In this way, the data set around a class is found and the data set plus the data in this class can be trained to form a classifier. With cooperative clustering, one binary classifier in the one-vs-all approach can be trained with far less samples. Furthermore, a kNN method is proposed to accelerate the classifying procedure. With this algorithm, both training and classification efficiency are improved with a slight impact on classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. The term “cooperative clustering” is used firstly in our earlier work at the year of 2006 [31], and Kashef et al. called their algorithm “cooperative clustering” in [18] at 2010. In fact, Kashef’s cooperative clustering is an ensemble technique of multiple clustering algorithm, like the “collaborative clustering” [12]. On the contrary, our cooperative clustering is a new clustering technique used in some specifical context.

References

  1. Anand R, Mehrotra KG, Mohan CK, Ranka S (1995) Efficient classification for multiclass problems using modular neural networks. IEEE Trans Neural Netw 6:117–124

    Article  Google Scholar 

  2. Bezdek JC, Reichherzer TR, Lim GS, Attikiouzel Y (1998) Multiple-prototype classifier design. IEEE Trans Syst Man Cybern B 28:67–79

    Article  Google Scholar 

  3. Blake CL, Merz CJ (1998) UCI repository of machine learning databases. Department of Information and Computer Science, University of California, Irvine

  4. Chawla N, Bowyer K, Hall L, Kegelmeyer WP (2002) Smote: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    MATH  Google Scholar 

  5. Chen JH, Chen CH (2004) Reducing SVM classification time using multiple mirror classifiers. IEEE Trans Syst Man Cybern B 34:1173–1183

    Article  Google Scholar 

  6. Cheong S, Oh SH, Lee SY (2004) Support vector machines with binary tree architecture for multi-class classification. Neural Inf Process-Lett Rev 2(3):47–51

    Google Scholar 

  7. Chou C-H, Lin C-C, Liu Y-H, Chang F (2006) A prototype classification method and its use in a hybrid solution for multiclass pattern recognition. Pattern Recognit 39:624–634

    Article  MATH  Google Scholar 

  8. Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inf Theory 13(1):21–27

    Article  MATH  Google Scholar 

  9. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines. Cambridge University Press, Cambridge

    Google Scholar 

  10. Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    MATH  Google Scholar 

  11. Fei B, Liu J (2006) Binary tree of SVM: a new fast multiclass training and classification algorithm. IEEE Trans Neural Netw 17(3):696–704

    Article  Google Scholar 

  12. Forestier G, Gançarski P, Wemmert C (2010) Collaborative clustering with background knowledge. Data Knowl Eng 69:211–228

    Article  Google Scholar 

  13. García-Pedrajas N, Ortiz-Boyer D (2006) Improve multiclass pattern recognition by the combination of two strategies. IEEE Trans Pattern Anal Mach Intell 28(6):1001–1006

    Article  Google Scholar 

  14. Hastie T, Tibshirani R (1998) Classification by pairwise coupling. Ann Stat 26(2):451–471

    Article  MathSciNet  MATH  Google Scholar 

  15. Hong X, Chen S, Harris CJ (2007) A kernel-based two-class classifier for imbalanced data sets. IEEE Trans Neural Netw 18(1):28–41

    Article  Google Scholar 

  16. Hsu C-W, Lin C-J (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Article  Google Scholar 

  17. Japkowicz N (2001) Supervised versus unsupervised binary-learning by feedforward neural networks. Mach Learn 42(1–2):97–122

    Article  MATH  Google Scholar 

  18. Kashef R, Kamel MS (2010) Cooperative clustering. Pattern Recognit 43:2315–2329

    Article  MATH  Google Scholar 

  19. Kohonen T (2001) Self-organizing maps. Springer, Berlin

    Book  MATH  Google Scholar 

  20. Kubat M, Matwin S (1997) Addressing the curse of imbalanced training sets: one-sided selection. In: Proceedings of the 14th international conference on machine learning, pp 179–186

  21. Kuncheva LI, Bezdek JC (1998) Nearest prototype classification: clustering, genetic algorithms, or random search. IEEE Trans Syst Man Cybern B 28:160–164

    Article  Google Scholar 

  22. MacQueen JB (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability, Berkeley, University of California Press, pp 281–297

  23. Maulik U, Bandyopadhyay S (2002) Performance evaluation of some clustering algorithms and validity indices. IEEE Trans Pattern Anal Mach Intell 24(12):1650–1654

    Article  Google Scholar 

  24. Mitra S, Banka H, Pedrycz W (2006) Rough-fuzzy collaborative clustering. IEEE Trans Syst Man Cybern B 36:795–805

    Article  Google Scholar 

  25. Mollineda RA, Ferri FJ, Vidal E (2003) A merge-based condensing strategy for multiple prototype classifiers. IEEE Trans Syst Man Cybern B 32:662–668

    Article  Google Scholar 

  26. Pedrycz W (2002) Collaborative fuzzy clustering. Pattern Recognit Lett 23(14):1675–1686

    Article  MATH  Google Scholar 

  27. Platt JC (1998) Sequential minimal optimization: a fast algorithm for training support vector machines. Microsoft Research, Technical Report MSR-TR-98-14

  28. Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGSVM’s for multiclass classification. Adv Neural Inf Process Syst 12:547–553

    Google Scholar 

  29. Schneider P, Biehl M, Hammer B (2009) Adaptive relevance matrices in learning vector quantization. Neural Comput 21(12):3532–3561

    Article  MathSciNet  MATH  Google Scholar 

  30. Schölkopf B, Platt BJC, Shawe-Taylor J, Smola AJ (2001) Estimating the support of a high-dimensional distribution. Neural Comput 13(7):1443–1471

    Article  MATH  Google Scholar 

  31. Tian S, Mu S, Yin C (2006) Cooperative clustering for training SVMs. In: Proceedings of the third international symposium on neural networks. Lecture notes in computer science, vol 3971, pp 962–967

  32. Turney PD (1995) Cost-sensitive classification: empirical evaluation of a hybrid genetic decision tree induction algorithm. J Artif Intell Res 2:369–409

    Google Scholar 

  33. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  34. Veenman CJ, Peinders MJT (2005) The nearest subclass classifier: a compromise between the nearest mean and nearest neighbor classifier. IEEE Trans Pattern Anal Mach Intell 27(9):1417–1429

    Article  Google Scholar 

  35. Wu G, Chang EY (2005) KBA: kernel boundary alignment considering imbalanced data distribution. IEEE Trans Knowl Data Eng 17(6):786–795

    Article  Google Scholar 

Download references

Acknowledgments

This work is supported by the National Natural Science Foundation of China (No. 61105056), the Fundamental Research Funds for the Central Universities, and Shandong Provincial Natural Science Foundation, China (No. ZR2012FM024). The authors would also like to thank our colleague professor Jian Yu for his helpful suggestions.

Author information

Authors and Affiliations

  1. School of Computer and Information Technology, Beijing Jiaotong University, Beijing, People’s Republic of China

    Chuanhuan Yin & Shengfeng Tian

  2. Datang International Power Generation Co., Ltd., Beijing, People’s Republic of China

    Xiang Zhao

  3. School of Computer and Information Engineering, Shandong Agriculture University, Taian, People’s Republic of China

    Shaomin Mu

Authors
  1. Chuanhuan Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaomin Mu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shengfeng Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chuanhuan Yin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yin, C., Zhao, X., Mu, S. et al. A Fast Multiclass Classification Algorithm Based on Cooperative Clustering. Neural Process Lett 38, 389–402 (2013). https://doi.org/10.1007/s11063-013-9278-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-013-9278-9

Keywords

Search

Navigation