Skip to main content
SpringerLink
Log in

Nesting Algorithm for Multi-Classification Problems

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Support vector machines (SVMs) are originally designed for binary classifications. As for multi-classifications, they are usually converted into binary ones. In the conventional multi-classifiable algorithms, One-against-One algorithm is a very power method. However, there exists a middle unclassifiable region. In order to overcome this drawback, a novel method called Nesting Algorithm is presented in this paper. Our ideas are as follows: firstly, construct the optimal hyperplanes based on One-against-One approach. Secondly, if there exist data points in the middle unclassifiable region, select them to construct the optimal hyperplanes with the same hyperparameters. Thirdly, repeat the second step until there are no data points in the unclassifiable region or the region is disappeared. In this paper, we also prove the validity of the proposed algorithm for unclassifiable region and give the computational complexity analysis of the method. In order to examine the training accuracy and the generalization performance of the proposed algorithm, One-against-One algorithm, fuzzy least square support vector machine (FLS-SVM) and the proposed algorithm are applied to five UCI datasets. The results show that the training accuracy of the proposed algorithm is higher than the others, and its generalization performance is also comparable with them.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Abe S (2003) Analysis of multiclass support vector machines. In: Proceeding of international conference on computational intelligence for modelling control and automation, Vienna, Austria, pp 385–396

  2. Angulo C, Parra X, Català A (2003) K-SVCR a support vector machine for multi-class classification. Neurocomputing 55:57–77

    Article  Google Scholar 

  3. Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Haussler D (ed) Proceeding 5th Annu, ACM workshop comput learning theory, pp 144–152

  4. Bredensteniner EJ, Bennett KP (1999) Multicategory classification by support vector machines. Comput Optim Appl 12:53–79

    Article  MathSciNet  Google Scholar 

  5. Fürnkranz J (2002) Round robin classification. J Mach Learn Res 2:721–747

    Article  MATH  MathSciNet  Google Scholar 

  6. Hao ZF et al (2005) Twi-map support vector machine for multi-classification problems. In: Proceeding of ISNN 2005 conference, lecture notes in computer science 3496. Chongqing, PT1, pp 869–874

  7. Hsu CW, Lin CJ (2002) A comparison of methods for multiclass support vector machines. IEEE Tran Neural Netw 13:415–425

    Article  Google Scholar 

  8. Inoue T, Abe S (2001) Fuzzy support vector machines for pattern classification. In Proceedings of international joint conference on neural networks (IJCNN‘01) 2:1449–1454

  9. KreBel UHG (1999) Pairwise classification and support vector machines. In: Schölkopf B, Burges CJ Smola AJ (eds) Advances in Kernel methods: support vector learning. The MIT Press, Cambridge, pp 255–268

  10. Liu B, Hao ZF, Yang XW (2006) Binary tree support vector machine based on kernel fisher discriminant for multi-classification. In: Proceeding of ISNN 2006 conference, lecture notes in computer science (in press)

  11. Murphy PM, Aha DW (1992) UCI repository of machine learning database

  12. Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Solla SA, Leen TK, Müller KR (eds) Advances in neural information processing systems 12. The MIT Press, Cambridge, pp 547–553

    Google Scholar 

  13. Rifkin R, Klautau A (2004) In defense of One-Vs-All classification. J Mach Learn Res 5:101–141

    Google Scholar 

  14. Saitoh S (1988) Theory of reproducing kernels and its applications. Longman, Harlow

    MATH  Google Scholar 

  15. Suykens JAK (2000) Least squares support vector machine for classification and nonlinear modeling. Neural Netw World 10(1–2): 29–47

    MathSciNet  Google Scholar 

  16. Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Netw 16:785–792

    Article  PubMed  Google Scholar 

  17. Vapnik VN (1995) The nature of statistical learning theory. Springer, London

    MATH  Google Scholar 

  18. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  19. Vapnik VN (1999) An overview of statistical learning theory. IEEE Trans Neural Netw 10(5):988–999

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Engineering, South China University of Technology, Guangzhou, 510640, People’s Republic of China

    Bo Liu

  2. School of Mathematical Science, South China University of Technology, Guangzhou, 510640, People’s Republic of China

    Zhifeng Hao & Xiaowei Yang

Authors
  1. Bo Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhifeng Hao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaowei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, B., Hao, Z. & Yang, X. Nesting Algorithm for Multi-Classification Problems. Soft Comput 11, 383–389 (2007). https://doi.org/10.1007/s00500-006-0093-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-006-0093-3

Keywords

Search

Navigation