Synergy of Logistic Regression and Support Vector Machine in Multiple-Class Classification

Abstract

In this paper, we focus on multiple-class classification problems. By using polytomous logistic regression and support vector machine together, we come out a hybrid multi-class classifier with very promising results in terms of classification accuracy. Usually, the multiple-class classifier can be built by using many binary classifiers as its construction bases. Those binary classifiers might be trained by either one-versus-one or one-versus-others manners, and the final classifier is constructed by some kinds of “leveraging” methods; such as majority vote, weighted vote, regression, etc. Here, we propose a new way for constructing binary classifiers, which might take the relationship of classes into consideration. For example, the level of severity of a disease in medial diagnostic. Depending on the methods used for constructing binary classifiers, the final classifier will be constructed/assembled by nominal, ordinal or even more sophisticated polytomous logistic regression techniques. This hybrid method has been apply to many real world bench mark data sets and the results shows that this new hybrid method is very promising and out-performs the classifiers using the technique of the support vector machine alone.