Realization of subjective correspondence in artificial neural network trained by Fahlman and Lebiere's learning algorithm

Abstract

In realizing surjective correspondence as an incremental learning by feedforward neural network system, it is desirable that the network designer be able to make use of hidden outputs in the alreadytrained network realizing injective correspondence in order to be adapted to a changeable environment that may demand learning of newly added patterns into the same category. To design a system that performs an extended task without destroying the hidden outputs gained by the previously trained network, some new hidden units are incorporated to acquire additional information required to realize the newly defined task. Fahlman and Lebiere's (FL) learning algorithm is particularly suitable for this purpose since this algorithm can gradually add the required number of new hidden units. Previous studies show that FL network generalizes far better than Backpropagation (BP) network [10], [11]. And it has also been reported that an extended FL network which realizes an incremental learning with increased category have generalization ability superior to BP network [12], [13]. In this paper, we describe a realization of surjective correspondence as an incremental learning by FL algorithm. Investigation shows that FL network trained surjective correspondence has better generalization ability than BP network due to the attainment of well-saturated hidden outputs in FL network.