Optimizing lattice-based associative memory networks by evolutionary algorithms
Section snippets
Associative memory networks
Associative memory networks (AMNs) are a class of artificial neural networks and have as widely known members the fuzzy and radial basis function [1] networks. All these models share the ability to store (in difference to other neural networks, such as the Multilayer Perceptron) information locally. This feature makes them suitable for instantaneous adaption by solving an overdetermined set of linear equations. If the input space is normalized by a lattice, AMNs are called lattice-based
B-splines
B-splines have been employed in surface-fitting algorithms for computer-aided design tasks [9] and they are recursively defined bywith some valuable characteristics such as
- •
Positivity:
- •
Local support:
- •
Partition of unity:
Evolutionary optimized LB-AMNs
The problem is to find a (sub)optimal LB-AMN by using only important inputs from a given input matrix and to build up a network topology with as few as possible free parameters which can handle the problem sufficiently. Each LB-AMN is defined by
- •
the used inputs (columns) of the input matrix;
- •
the order(s) of the basis functions that cover the input(s);
- •
the number of basis functions used to cover the input(s);
- •
the knotpositions which define the shape of the basis functions.
To find (sub)optimal values
A simple mathematical problem
To take a first look on the capability of our approach we optimize the topology of a B-spline LB-AMN for a one-dimensional mathematical function given byUsing 201 training patterns in the interval [−1,1] and parameters as given below the (sub)optimal B-spline LB-AMN was calculated. A test error was not computed since the use of maximal 12 weights to model non-linear data with two discontinuities will retrench overfitting. This example should only
Real world toxicity data
Considering a difficult real world problem we examined toxicity data. The toxicity dataset provided by [14] consists of 211 pesticides from six different chemical classes with data on toxicity for Daphnia Magna (a water flea). Each molecule is described by about 160 descriptors which were used as system input and the desired output is the toxicity for Daphnia Magna (Fig. 8). For further information about the dataset see [15].
Each molecule of the dataset is classified up to three different
Conclusion
We presented an approach for problem specific structure identification optimization of LB-AMNs with the possibility to use nearly arbitrary basis functions. The algorithm performs well and for high-dimensional problems it is possible to detect relevant inputs. Furthermore the optimal number and shape of the basis functions can be found. More research will be done on accelerating the genetic algorithm by an “intelligent” control system, considering generalized basis functions and by constructing
Acknowledgements
This work has been supported by the Commission of the European Communities under the Program “Environment and Climate”, Project “COMET”, Contract No. ENV4-CT97-0508.
References (15)
- et al.
Multivariable functional interpolation and adaptive networks
Complex Systems
(1988) A new approach to manipulator control: the cerebellar model articulation controller (CMAC)
Trans. ASME Jnl. Dyn. Sys. Meas. and Control
(1975)- J. Moody, Fast learning in multi-resolution hierachies, in: D.S. Touretzky (Ed.), Advances in Neural Information...
Sparse Distributed Memory
(1988)- P. Kanerva, Associative-memory models of the cerebellum, in: I. Aleksander, J. Tayler (Eds.), Artificial Neural...
- et al.
Symetric decomposition of positive bandmatrices
Numerische Mathematik
(1965) Numerical Recipes in C: The Art of Scientific Computing
(1996)
Cited by (2)
Data driven modeling using descriptive Takagi-Sugeno fuzzy systems
2003, Recent Advances in Intelligent Systems and Signal ProcessingEvolutionary system identification via descriptive Takagi Sugeno fuzzy systems
2003, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)