Learning Target Selection in Creating Negatively Correlated Neural Networks

Abstract

In an ensemble system with a number of learning modules, it is important for these modules to be cooperative in solving a given task. One measurement on their cooperation is to ensure that these learning modules should be negatively correlated while the expected mean squared errors of the ensemble with these negatively correlated modules should be as small as possible. This paper summarizes three different approaches to how to set target values on the given data so that negative correlation learning could be more effective in creating negatively correlated neural networks. Rather than fixing the learning targets in the whole learning process, it would be better to adaptively select different learning targets among individual networks in the ensemble system so that all the individual networks could learn to be cooperative through the interactive learning. The first two approaches specified the target values based on either the ensemble’s behavior or the individual module’s behavior. The third approach introduced the different targets between two randomly sampled data sets.