An Incremental Learning Algorithm for Optimizing High-Dimensional ANN-Based Classification Systems

Abstract

This paper presents an incremental learning algorithm primarily intended for adjusting and optimizing high dimensional backpropagation ANN-based supervised classification systems. The algorithm avoids the highly time-consuming pre-processing stage used to reduce dimensionality through the deletion or averaging of redundant information and the establishment of an appropriate processing window. The proposed algorithm acts during the training process of the ANN by automatically obtaining the optimal window size and transformation parameter values needed for a given set of classification requirements. During this process, it changes the network topology on line by adaptively appending input units and their corresponding connections to the existing network. The proposed process allows the ANN to learn incrementally by adapting to the new topology without forgetting what had been learnt earlier. This process could also be used as an incremental learning system in, for instance, robotic systems; when new instances are fed, as it would not need to perform a whole training stage. Instead, the knowledge encoded in the new instances could be learnt through the proposed adjustment of network topology.