Skip to main content
SpringerLink
Log in

Fast Multiclass SVM Classification Using Decision Tree Based One-Against-All Method

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

We present an improved version of One-Against-All (OAA) method for multiclass SVM classification based on a decision tree approach. The proposed decision tree based OAA (DT-OAA) is aimed at increasing the classification speed of OAA by using posterior probability estimates of binary SVM outputs. DT-OAA decreases the average number of binary SVM tests required in testing phase to a greater extent when compared to OAA and other multiclass SVM methods. For a balanced multiclass dataset with K classes, under best situation, DT-OAA requires only (K + 1)/2 binary tests on an average as opposed to K binary tests in OAA; however, on imbalanced multiclass datasets we observed DT-OAA to be much faster with proper selection of order in which the binary SVMs are arranged in the decision tree. Computational comparisons on publicly available datasets indicate that the proposed method can achieve almost the same classification accuracy as that of OAA, but is much faster in decision making.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Vapnik V (1998) Statistical learning theory. Wiley, New York

    MATH  Google Scholar 

  2. Burges CJC (1996) Simplified support vector decision rules. In: Proceedings of the 13th international conference on machine learning, Italy. Morgan Kaufmann, San Francisco, CA, pp 71–77

  3. Downs T, Gates K, Masters A (2001) Exact simplification of support vector solutions. Journal of Machine Learning Research 2: 293–297

    Article  Google Scholar 

  4. Osuna E, Girosi F (1999) Reducing the run-time complexity of support vector machines. Advances in kernel methods—support vector learning. MIT Press, Cambridge, pp 271–283

    Google Scholar 

  5. Li Q, Jiao L, Hao Y (2007) Adaptive simplification of solution for support vector machines. Pattern Recognit 40: 972–980

    Article  MATH  Google Scholar 

  6. Zhang K, Kwok JT (2010) Simplifying mixture models through function approximation. IEEE Trans Neural Netw 21(4): 644–658

    Article  Google Scholar 

  7. Tang B, Mazzoni D (2006) Multiclass reduced set support vector machines. In: Proceedings of the 23rd international conference on machine learning. Pittsburgh, PA, pp 921–928

  8. Knerr S, Personnaz L, Dreyfus G (1990) Single-layer learning revisited a stepwise procedure for building and training a neural network. In: Fogelman J (eds) Neurocomputing: algorithms, architechtures, and applications. Springer-Verlag, New York

    Google Scholar 

  9. Hastie T, Tibshirani R (1998) Classification by pairwise coupling. Ann Stat 26(2): 451–471

    Article  MATH  MathSciNet  Google Scholar 

  10. Platt JC, Christiani N, Shawe–Taylor (1999) Large margin DAGs for multiclass classification. In: Solla SA, Leen TK, Muller KR (eds) Proceedings of the neural information processing systems (NIPS’99). MIT Press, pp 547–553

  11. Fei B, Liu J (2006) Binary tree of SVM: a new fast multiclass training and classification algorithm. IEEE Trans Neural Netw 17(3): 696–704

    Article  Google Scholar 

  12. Pedrajas NG, Boyer DO (2006) Improving multiclass pattern recognition by the combination of two strategies. IEEE Trans Pattern Anal Mach Intell 28(6): 1001–1006

    Article  Google Scholar 

  13. Hsu C, Lin C (2001) A comparison of methods for multi-class support vector machines. IEEE Trans Neural Netw 13(2): 415–425

    Google Scholar 

  14. Rifkin R, Klautau A (2004) In defense of one-vs-all classification. J Mach Learn Res 5: 101–141

    MathSciNet  Google Scholar 

  15. Mangasarian OL (1994) Nonlinear programming. SIAM, Philadelphia

    MATH  Google Scholar 

  16. Frank A, Asuncion A (2010) UCI machine learning repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.

  17. Statlog datasets available at ftp://ncc.up.pt/pub/statlog/

  18. Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5): 550–554

    Article  Google Scholar 

  19. Joachims T (1998) Making large-scale SVM learning practical. Advances in kernel methods—support vector learning. MIT Press, Cambridge

    Google Scholar 

  20. MATLAB. http://www/mathworks.com

Download references

Author information

Authors and Affiliations

  1. Control Group, Department of Electrical Engineering, Indian Institute of Technology Delhi, Hauz Khas, New Delhi, 110016, India

    M. Arun Kumar & M. Gopal

Authors
  1. M. Arun Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Gopal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to M. Arun Kumar.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Arun Kumar, M., Gopal, M. Fast Multiclass SVM Classification Using Decision Tree Based One-Against-All Method. Neural Process Lett 32, 311–323 (2010). https://doi.org/10.1007/s11063-010-9160-y

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-010-9160-y

Keywords

Search

Navigation