Optimized learning for improving the evolution of piecewise linear separation incremental algorithms

Abstract

In this paper we address the problems which may appear when using the classical Perceptron or Pocket algorithms in order to train the units generated by Piecewise Linear Separation (PLS) incremental algorithms. These problems are due to the type of optimal solutions found by such training algorithms. Some of these solutions force a useless separation of input data, resulting in that the new units added to the network by the incremental algorithm are again faced with the same problem. The final network would then be composed of a large number of redundant units, each of them trying to solve exactly the same problem and arriving at exactly the same solution. We review some modifications proposed for improving the training algorithms, which are mainly based on the evaluation of entropy-like functions calculated for the input distributions. Furthermore, an alternative solution is proposed which has the advantage of the low computational cost associated to it. This method compares well, as simulation results show, with the methods based on Information Theory concepts.

Holder of an FI research Grant under the Generalitat de Catalunya's Educ. Dept.