Concept Updating with Support Vector Machines

Abstract

In many practical situations in inductive learning algorithms, it is often expected to further improve the generalization capability after the learning process has been completed if new data are available. One of the common approaches is to add training data to the learning algorithm and retrain it, but retraining for each new data point or data set can be very expensive. In view of the learning methods of human beings, it seems natural to build posterior learning results upon prior results. In this paper, we apply Support Vector Machine(SVM) to the concept updating procedure. If initial concept would be built up by inductive algorithm, then concept updated is the normal solution corresponding to the initial concept learned. It was shown that concept learned would not change if the new available data located in error-insensitive zone. Especially, concept initially learned and updated by SVR induces an incremental SVR approximately learning method for large scale data. We tested our method on toys data sets and 7 regression bench mark data set. It shown that generalization capacity after updating with SVR was improved according to FVU or MSE on the independent test set.