Authors:
Jiri Grim
1
and
Pavel Pudil
2
Affiliations:
1
Academy of Sciences of the Czech Republic, Czech Republic
;
2
Prague University of Economics, Czech Republic
Keyword(s):
Probabilistic Neural Networks, Product Mixtures, Mixtures of Dependence Trees, EM Algorithm.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Artificial Intelligence and Decision Support Systems
;
Biomedical Engineering
;
Biomedical Signal Processing
;
Computational Intelligence
;
Data Manipulation
;
Enterprise Information Systems
;
Health Engineering and Technology Applications
;
Human-Computer Interaction
;
Methodologies and Methods
;
Modular Implementation of Artificial Neural Networks
;
Neural Network Software and Applications
;
Neural Networks
;
Neurocomputing
;
Neurotechnology, Electronics and Informatics
;
Pattern Recognition
;
Physiological Computing Systems
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Theory and Methods
Abstract:
We compare two probabilistic approaches to neural networks - the first one based on the mixtures of product
components and the second one using the mixtures of dependence-tree distributions. The product mixture
models can be efficiently estimated from data by means of EM algorithm and have some practically important
properties. However, in some cases the simplicity of product components could appear too restrictive and a
natural idea is to use a more complex mixture of dependence-tree distributions. By considering the concept of
dependence tree we can explicitly describe the statistical relationships between pairs of variables at the level
of individual components and therefore the approximation power of the resulting mixture may essentially
increase. Nonetheless, in application to classification of numerals we have found that both models perform
comparably and the contribution of the dependence-tree structures decreases in the course of EM iterations.
Thus the optimal estimate of th
e dependence-tree mixture tends to converge to a simple product mixture model.
Regardless of computational aspects, the dependence-tree mixtures could help to clarify the role of dendritic
branching in the highly selective excitability of neurons.
(More)