Mutual Information Neural Networks: A New Connectionist Paradigm for Dynamic Pattern Recognition Tasks

Abstract

This paper presents a new probabilistic neural network paradigm for dynamic pattern recognition problems. The new approach includes the following innovations: 1) It is based on a self-organizing learning approach using information theory principles 2) The neuron activations are interpreted as probabilities and represent a probabilistic decision boundary in the feature space 3) A combination of unsupervised and supervised learning algorithms can be used to train the network weights 4) The neuron probabilities can be further refined by corrective training methods leading to a joint optimization of both, the weights and the neuron probabilities 5) The neural network can process dynamic, time varying patterns of arbitrary length 6) The output activations of the neural network can be evaluated directly or optionally treated as the input to other probabilistic pattern recognition algorithms, e.g. Hidden Markov Models. This combination leads to a powerful hybrid pattern recognition approach.