Base Vector Selection for Support Vector Machine

Abstract

SVM has been receiving increasing interest in areas ranging from its original application in pattern recognition to other applications such as regression estimation due to its remarkable generalization performance. However, it also contains some defects such as storage problem (in training process) and sparsity problem. In this paper, a new method is proposed to pre-select the base vectors from the original data according to vector correlation principle, which could greatly reduce the scale of the optimization problem and improve the sparsity of the solution. The method could capture the structure of the data space by approximating a basis of the subspace of the data; therefore, the statistical information of the training samples is preserved. In the paper, the process of mathematical deduction is given in details and results of simulations on artificial data and practical data have been done to validate the performance of base vector selection (BVS) algorithm. The experimental results show the combination of such algorithm with SVM can make great progress while can’t sacrifice the SVM’s performance.