Skip to main content
SpringerLink
Log in

Multiclass classifier based on boundary complexity

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

This paper presents several criteria for partition of classes for the support vector machine based hierarchical classification. Our clustering algorithm combines support vector machine and binary tree, it is a divisive (top-down) approach in which a set of classes is automatically separated into two smaller groups at each node of the hierarchy, it splits the classes based on the normalized cuts clustering algorithm. Our clustering algorithm considers the involved classes rather than the individual data samples. In the new proposed measures, similarity between classes is determined based on boundary complexity. In these measures, concepts such as the upper bound of error and Kolmogorov complexity are used. We reported results on several data sets and five distance/similarity measures. Experimental results demonstrate the superiority of the proposed measures compared to other measures; even when applied to nonlinearly separable data, the new criteria perform well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Breiman L, Friedman J, Olshen RA, Stone CJ (1984) Classification and regression trees. Chapman and Hall, London

    MATH  Google Scholar 

  2. Fei B, Liu J (2006) Binary tree of SVM: a new fast multiclass training and classification algorithm. IEEE Trans Neural Netw 17(3):696–704

    Article  Google Scholar 

  3. Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, Oxford

    Google Scholar 

  4. Knerr S (1990) Single-Layer learning revissted: a stepwise procedure for building and training a neural network, Neurocomputing: Algorithms, Architectures and applications. NATO ASI series, vol F68, Springer, pp 41–50

  5. Bay SD (1998) Combining nearest neighbor classifiers through multiple feature subsets. In: Proceedings of the 17th international conference on machine learning. Madison, WI, pp 37–45

  6. Irina Rish (2001) An empirical study of the naive Bayes classifier. In: IJCAI workshop on empirical methods in artificial intelligence

  7. Changa C–C, Chien L-J, Lee Y-J (2011) A novel framework for multi-class classification via ternary smooth support vector machine. Pattern Recognit 44(6):1235–1244

    Article  Google Scholar 

  8. Allwein E, Schapire R, Singer Y, Kaelbling P (2000) Reducing multiclass to binary: a unifying approach for margin classifiers. J Mach Learn Res 1:113–141

    Google Scholar 

  9. Duan KB and Keerthi SS (2005) Which is the best multiclass svm method? An empirical study. In: Proceedings of the sixth international workshop on multiple classifier systems, pp 278–285

  10. Garcıa S, Herrera F (2008) An extension on “statistical comparisons of classifiers over multiple data sets” for all pairwise comparisons. J Mach Learn Res 9:2677–2694

    MATH  Google Scholar 

  11. Hsu C-W, Lin C-J (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Netw 13(2):415–425

    Google Scholar 

  12. Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Learn Res 5:101–141

    MATH  MathSciNet  Google Scholar 

  13. Galar M, Fenandez A, Barrenechea E (2011) An overview of ensemble methods for binary classifiers in multi-class problems: experimental study on one-vs-one and one-vs-all schemes. Pattern Recognit 44:1761–1776

    Article  Google Scholar 

  14. Samuel R.R (2010) Model combination in multiclass classification, PhD thesis, University of Colorado

  15. Hastie T, Tibshirani R, Friedman J (2003) The elements of statistical learning. Springer, Berlin

    Google Scholar 

  16. Lorena vA (2008) Review on the combination of binary classifiers in multiclass problems. Artif Intell Rev 30(1–4):19–37

    Article  Google Scholar 

  17. Dietterich TG, Bakiri G (1995) Solving multiclass learning problems via error-correcting output codes. J Artif Intell Res 2:263–286

    MATH  Google Scholar 

  18. Ou G, Murphy YL (2007) Multi-class pattern classification using neural networks. Pattern Recognit 40(1):4–18

    Article  MATH  Google Scholar 

  19. Vural V and Dy JG (2004) A hierarchical method for multi-class support vector machines. In: Proceedings of the twenty-first international conference machine learning

  20. Casasent D, Wang Y-C (2005) A hierarchical classifier using new support vector machines for automatic target recognition. Neural Netw 18:541–548

    Article  Google Scholar 

  21. Chen Y, Crawford MM, Ghosh J (2004) Integrating support vector machines in a hierarchical output space decomposition framework. In: Proceedings of the international geoscience remote sensing Symposium, pp 949–952

  22. Griffin G, Perona P (2008) Learning and using taxonomies for fast visual categorization. IEEE computational society conference on computer vision and pattern recognition

  23. Zhigang L, Wenzhong S, Qianqing Q, Xiaowen L, Donghu X (2005) Hierarchical support vector machines. Geosci. Remote Sensing Symp

  24. Yan D, Huang D, Jordan MI (2009) Fast approximate spectral clustering. In: Proc. ACM Int. Conf. Knowl. Disc. Data Mining, Paris, France

  25. Lu JF, Tang JB, Tang ZM, Yang JY (2008) Hierarchical initialization approach for k-means clustering. Pattern Recognit Lett 29:787–795

    Article  Google Scholar 

  26. Loewenstein Y, Portugaly E, Fromer M, Linial M (2008) Efficient algorithms for accurate hierarchical clustering of huge datasets: tackling the entire protein space. Bioinformatics 24(13):i41–i49

    Article  Google Scholar 

  27. Hamidzadeh J, Monsefi R, Sadoghi Yazdi H (2012) DDC: distance-based decision classifier. Neural Comput Appl 21:1697–1707

    Article  Google Scholar 

  28. Vural V and Dy JG (2004) A hierarchical method for multi-class support vector machines. In: Proceedings of the twenty-first international conference on machine learning

  29. Chen Y, Crawford M.M, Ghosh J (2004) Integrating support vector machines in a hierarchical output space decomposition framework. In: Proceedings of the International on Geoscience and Remote Sensing Symposium, pp 949–952

  30. Griffin G, Perona P (2008) Learning and using taxonomies for fast visual categorization. IEEE Comput. Soc. Conf. on Computer Vision and Pattern Recognition

  31. Cevikalp H (2010) New clustering algorithms for the support vector machine based hierarchical classification. Pattern Recognit Lett 31(11):1285–1291

    Article  Google Scholar 

  32. Cortes C and Vapnik V (2002) Support-vector networks. Mach Learn 273–297

  33. Basu M, Ho TK (2006) Data complexity in pattern recognition. Springer, Berlin

    Book  MATH  Google Scholar 

  34. Li L (2006) Data complexity in machine learning and novel classification algorithms, PhD thesis, California Institute of Technology

  35. Furnkranz J (2006) Round robin classification. J Mach Learn Res 2:721–747

    MathSciNet  Google Scholar 

  36. Pelckmans K, Suykens J.A.K, Van Gestel T, De Brabanter J (2003) Ls-SVM: alibrary for support vector machines http://www.esat.kuleuven.ac.be/sista/lssvmlab/

  37. Michie D, Spiegelhalter D. J and Taylor C. C (1994) Machine Learning, Neural and Statistical Classification [Online]. Available:ftp.ncc.up.pt/pub/statlog/

  38. Asuncion A, Newman D, UCImachinelearningrepository (2007) URL:http://www.ics.uci.edu/_mlearn/MLRepository.html

  39. Hullermeier E, Vanderlooy S (2010) Combing predictions in pairwise classification: an optimal adaptive voting strategy and its relation to weighted voting. Pattern Recognit 43:128–142

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Department, Ferdowsi University of Mashhad, Mashhad, Iran

    Hamid Reza Ghaffari & Hadi Sadoghi Yazdi

  2. Center of Excellence on Soft Computing and Intelligent Information Processing, Ferdowsi University of Mashhad, Mashhad, Iran

    Hadi Sadoghi Yazdi

Authors
  1. Hamid Reza Ghaffari
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hadi Sadoghi Yazdi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamid Reza Ghaffari.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Ghaffari, H.R., Sadoghi Yazdi, H. Multiclass classifier based on boundary complexity. Neural Comput & Applic 24, 985–993 (2014). https://doi.org/10.1007/s00521-012-1303-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-012-1303-9

Keywords

Search

Navigation