Abstract
The support vector machine (SVM) has a high generalisation ability to solve binary classification problems, but its extension to multi-class problems is still an ongoing research issue. Among the existing multi-class SVM methods, the one-against-one method is one of the most suitable methods for practical use. This paper presents a new multi-class SVM method that can reduce the number of hyperplanes of the one-against-one method and thus it returns fewer support vectors. The proposed algorithm works as follows. While producing the boundary of a class, no more hyperplanes are constructed if the discriminating hyperplanes of neighbouring classes happen to separate the rest of the classes. We present a large number of experiments that show that the training time of the proposed method is the least among the existing multi-class SVM methods. The experimental results also show that the testing time of the proposed method is less than that of the one-against-one method because of the reduction of hyperplanes and support vectors. The proposed method can resolve unclassifiable regions and alleviate the over-fitting problem in a much better way than the one-against-one method by reducing the number of hyperplanes. We also present a direct acyclic graph SVM (DAGSVM) based testing methodology that improves the testing time of the DAGSVM method.
Similar content being viewed by others
References
Cristanni N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, Cambridge, UK
Vapnik VN (1998) Statistical learning theory. Wiley, New York
Kreßel UH-G (1998) Pairwise classification and support vector machines. In: Schölkopf B, Burges C, Somla A (eds) Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 255–268
Platt JC, Cristianini N, Shawe-Taylor J (2000) Large margin DAGs for multiclass classification. In: Advances in neural information processing systems, vol 12, MIT Press, Cambridge, MA, pp 547–553
Hsu C-W, Lin C-J (2002) A comparison of methods for multiclass support vector machines. IEEE Trans Neural Networks, 13:415–425
Weston J, Watkins C (1998) Multi-class support vector machines. Technical Report CSD-TR-98–04, Royal Holloway, University of London
Crammer K, Singer Y (2000) On the learnability and design of output codes for multiclass problems. In: Proceedings of the 13th Annual Conference on Computational learning theory, 28 June–1 July 2000, pp 35–46
Lee Y, Lin Y, Wahba G (2001) Multicategory support vector machines. Technical Report No. 1043, University of Wisconsin
Hastie T, Tibshirani R (1998) Classification by pairwise coupling. In: Jordan MI, Kearnsa MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge, MA
Debnath R, Takahide N, Takahashi H (2003) A fast learning decision-based SVM for multi-class problems. In: Proceedings of the International Conference on Machine learning and applications, Los Angeles, CA, USA, 23–24 June 2003, pp 128–134
Shawe-Taylor J, Cristianini N (1998) Data-dependent structral risk minimization for perceptron decision trees. In: Jordan MI, Kearnsa MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge, MA, pp 336–342
Freund Y, Iyer R, Schapire RE, Singer Y (1998) An efficient boosting algorithm for combining preferences. In: Proceedings of the 15th International Conference on Machine learning, 24–26 July 1998, Madison, WI
Opper M, Urbanczik R (2001) Universal learning curves of support vector machines. Phys Rev Lett 86:4410–4413
Blake CL, Merz CJ (1998) UCI repository of machine learning databases. University of California Department Information and Computer Science, Irvine, CA. Online: available at http://www.ics.uci.edu/~mlearn/MLRepository.html
Tsujinishi D, Abe S (2003) Fuzzy least squares support vector machines for multiclass problems. Neural Networks 16:785–792
Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schölkopf B, Burges C, Somla A (eds) Advances in kernel methods: support vector machine. MIT Press, Cambridge, MA, pp 185–208
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Debnath, R., Takahide, N. & Takahashi, H. A decision based one-against-one method for multi-class support vector machine. Pattern Anal Applic 7, 164–175 (2004). https://doi.org/10.1007/s10044-004-0213-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10044-004-0213-6