Skip to main content
SpringerLink
Log in

A decision based one-against-one method for multi-class support vector machine

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

The support vector machine (SVM) has a high generalisation ability to solve binary classification problems, but its extension to multi-class problems is still an ongoing research issue. Among the existing multi-class SVM methods, the one-against-one method is one of the most suitable methods for practical use. This paper presents a new multi-class SVM method that can reduce the number of hyperplanes of the one-against-one method and thus it returns fewer support vectors. The proposed algorithm works as follows. While producing the boundary of a class, no more hyperplanes are constructed if the discriminating hyperplanes of neighbouring classes happen to separate the rest of the classes. We present a large number of experiments that show that the training time of the proposed method is the least among the existing multi-class SVM methods. The experimental results also show that the testing time of the proposed method is less than that of the one-against-one method because of the reduction of hyperplanes and support vectors. The proposed method can resolve unclassifiable regions and alleviate the over-fitting problem in a much better way than the one-against-one method by reducing the number of hyperplanes. We also present a direct acyclic graph SVM (DAGSVM) based testing methodology that improves the testing time of the DAGSVM method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Cristanni N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge, UK

  2. Vapnik VN (1998) Statistical learning theory. Wiley, New York

  3. Kreßel UH-G (1998) Pairwise classification and support vector machines. In: Schölkopf B, Burges C, Somla A (eds) Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 255–268

  4. Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Advances in neural information processing systems, vol 12, MIT Press, Cambridge, MA, pp 547–553

  5. Hsu C-W, Lin C-J (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Networks, 13:415–425

    Google Scholar 

  6. Weston J, Watkins C (1998) Multi-class support vector machines. Technical Report CSD-TR-98–04, Royal Holloway, University of London

  7. Crammer K, Singer Y (2000) On the learnability and design of output codes for multiclass problems. In: Proceedings of the 13th Annual Conference on Computational learning theory, 28 June–1 July 2000, pp 35–46

  8. Lee Y, Lin Y, Wahba G (2001) Multicategory support vector machines. Technical Report No. 1043, University of Wisconsin

  9. Hastie T, Tibshirani R (1998) Classification by pairwise coupling. In: Jordan MI, Kearnsa MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge, MA

  10. Debnath R, Takahide N, Takahashi H (2003) A fast learning decision-based SVM for multi-class problems. In: Proceedings of the International Conference on Machine learning and applications, Los Angeles, CA, USA, 23–24 June 2003, pp 128–134

  11. Shawe-Taylor J, Cristianini N (1998) Data-dependent structral risk minimization for perceptron decision trees. In: Jordan MI, Kearnsa MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge, MA, pp 336–342

  12. Freund Y, Iyer R, Schapire RE, Singer Y (1998) An efficient boosting algorithm for combining preferences. In: Proceedings of the 15th International Conference on Machine learning, 24–26 July 1998, Madison, WI

  13. Opper M, Urbanczik R (2001) Universal learning curves of support vector machines. Phys Rev Lett 86:4410–4413

    Article  Google Scholar 

  14. Blake CL, Merz CJ (1998) UCI repository of machine learning databases. University of California Department Information and Computer Science, Irvine, CA. Online: available at http://www.ics.uci.edu/~mlearn/MLRepository.html

  15. Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Networks 16:785–792

    Article  Google Scholar 

  16. Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B, Burges C, Somla A (eds) Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 185–208

Download references

Author information

Authors and Affiliations

  1. Department of Information and Communication Engineering, The University of Electro-Communications, 1-5-1 Chofugaoka, Chofu-shi, 182-8585, Tokyo, Japan

    R. Debnath, N. Takahide & H. Takahashi

Authors
  1. R. Debnath
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. N. Takahide
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. H. Takahashi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to R. Debnath.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Debnath, R., Takahide, N. & Takahashi, H. A decision based one-against-one method for multi-class support vector machine. Pattern Anal Applic 7, 164–175 (2004). https://doi.org/10.1007/s10044-004-0213-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-004-0213-6

Keywords

Search

Navigation