Abstract
Posterior probability support vector machines (PPSVMs) prove robust against noises and outliers and need fewer storage support vectors (SVs). Gonen et al. (2008) extended PPSVMs to a multiclass case by both single-machine and multimachine approaches. However, these extensions suffer from low classification efficiency, high computational burden, and more importantly, unclassifiable regions. To achieve higher classification efficiency and accuracy with fewer SVs, a binary tree of PPSVMs for the multiclass classification problem is proposed in this letter. Moreover, a Fisher ratio separability measure is adopted to determine the tree structure. Several experiments on handwritten recognition datasets are included to illustrate the proposed approach. Specifically, the Fisher ratio separability accelerated binary tree of PPSVMs obtains overall test accuracy, if not higher than, at least comparable to those of other multiclass algorithms, while using significantly fewer SVs and much less test time.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cortes, C., Vapnik, V., 1995. Support-vector networks. Mach. Learn., 20(3):273–297. [doi:10.1007/BF00994018]
Dietterich, T.G., Bakiri, G., 1995. Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res., 2(1):263–286.
Duda, R.O., Har, P.E., 1973. Pattern Classification and Scene Analysis. Wiley, New York.
Fei, B., Liu, J., 2006. Binary tree of SVM: a new fast multiclass training and classification algorithm. IEEE Trans. Neur. Netw., 17(3):696–704. [doi:10.1109/TNN.2006.872343]
Gonen, M., Tanuğur, A.G., Alpaydin, E., 2008. Multiclass posterior probability support vector machines. IEEE Trans. Neur. Netw., 19(1):130–139. [doi:10.1109/TNN.2007.903157]
Guo, G., Li, S.Z., Chan, K.L., 2001. Support vector machines for face recognition. Image Vis. Comput., 19(9–10):631–638. [doi:10.1016/S0262-8856(01)00046-4]
Hsu, C.W., Lin, C.J., 2002. A comparison of methods for multi-class support vector machines. IEEE Trans. Neur. Netw., 13(2):415–425. [doi:10.1109/72.991427]
Hu, Z.H., Cai, Y.Z., Li, Y.G., Xu, X.M., 2005. Data fusion for fault diagnosis using multi-class support vector machines. J. Zhejiang Univ.-Sci., 6A(10):1030–1039. [doi:10.1631/jzus.2005.A1030]
Huang, P., Zhu, J., 2010. Multi-instance learning for software quality estimation in object-oriented systems: a case study. J. Zhejiang Univ.-Sci. C (Comput & Electron.), 11(2): 130–138. [doi:10.1631/jzus.C0910084]
KreBel, U.H.G., 1999. Pairwise classification and support vector machine. In: Schölkopf, B., Burges, C.J., Smola, A.J. (Eds.), Advances in Kernel Methods: Support Vector Learning. MIT Press, Cambridge, MA.
Leng, B., Qin, Z., Li, L.Q., 2007. Support vector machines active learning for 3D model retrieval. J. Zhejiang Univ.-Sci. A, 8(12):1953–1961. [doi:10.1631/jzus.2007.A1953]
Muller, K.R., Mika, S., Ratisch, G., Tsuda, K., Scholkopf, B., 2001. An introduction to kernel-based learning algorithms. IEEE Trans. Neur. Netw., 12(2):181–201. [doi:10. 1109/72.914517]
Platt, J., Cristianini, N., Shawe-Taylor, J., 2000. Large margin DAGSVM’s for multiclass classification. Adv. Neur. Inform. Process. Syst., 12:547–553.
Takahashi, F., Abe, S., 2002. Decision-Tree-Based Multiclass Support Vector Machine. Proc. 9th Int. Conf. on Neural Information, p.1418–1422.
Tao, Q., Wu, G.W., Wang, F.Y., Wang, J., 2005. Posterior probability support vector machines for unbalanced data. IEEE Trans. Neur. Netw., 16(6):1561–1573. [doi:10.1109/TNN.2005.857955]
Vapnik, V., 1995. The Nature of Statistical Learning Theory. Springer Verlag, New York.
Vapnik, V., 1998. Statistical Learning Theory. John Wiley & Sons, New York.
Xin, D., Wu, Z.H., Pan, Y.H., 2002. Probability output of multi-class support vector machines. J. Zhejiang Univ.-Sci., 3(2):131–134. [doi:10.1631/jzus.2002.0131]
Zhang, L., Zhou, W.D., Su, T.T., Jiao, L.C., 2007. Decision tree support vector machine. Int. J. Artif. Intell. Tools, 16(1):1–15. [doi:10.1142/S0218213007003163]
Author information
Authors and Affiliations
Corresponding author
Additional information
Project (Nos. 60874104 and 70971020) supported by the National Natural Science Foundation of China
Rights and permissions
About this article
Cite this article
Wang, Dl., Zheng, Jg. & Zhou, Y. Binary tree of posterior probability support vector machines. J. Zhejiang Univ. - Sci. C 12, 83–87 (2011). https://doi.org/10.1631/jzus.C1000022
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1631/jzus.C1000022