Multiobjective Multiclass Soft-Margin Support Vector Machine and Its Solving Technique Based on Benson’s Method

Abstract

In this paper, we focus on the all together model, which is one of the support vector machine (SVM) using a piece-wise linear function for multiclass classification. We already proposed a multiobjective hard-margin SVM model as a new all together model for piecewise linearly separable data, which maximizes all of the geometric margins simultaneously for the generalization ability. In addition, we derived a single-objective convex problem and showed that a Pareto optimal solution for the proposed multiobjective SVM is obtained by solving single-objective problems. However, in the real-world classification problem the data are often piecewise linearly inseparable. Therefore, in this paper we extend the hard-margin SVM for the data by using penalty functions for the margin slack variables between outliers and the corresponding discriminant hyperplane. Those functions are incorporated into the objective functions. Moreover, we derive a single-objective second-order cone programming (SOCP) problem based on Benson’s method and some techniques, and show that a Pareto optimal solution for the proposed soft-margin SVM is obtained by solving the SOCP iteratively. Furthermore through numerical experiments we verify that the proposed iterative method maximizes the geometric margins and constructs a classifier with a high generalization ability.