Optimizing lattice-based associative memory networks by evolutionary algorithms

https://doi.org/10.1016/S0020-0255(01)00142-6Get rights and content

Abstract

Problem specific network structure optimization subsumes the problem of input selection and network topology identification. Requirements to the network should be accuracy and good generalization abilities. In this contribution we describe in detail an evolutionary algorithm which performs both tasks well. Furthermore, approximation results on mathematical and real world data are presented. In this case we used lattice-based associative memory networks (LB-AMNs) using B-splines as basis functions. The method here is not restricted to B-splines as basis functions. The proposed method and algorithm can be seen as optimized classification system.

Section snippets

Associative memory networks

Associative memory networks (AMNs) are a class of artificial neural networks and have as widely known members the fuzzy and radial basis function [1] networks. All these models share the ability to store (in difference to other neural networks, such as the Multilayer Perceptron) information locally. This feature makes them suitable for instantaneous adaption by solving an overdetermined set of linear equations. If the input space is normalized by a lattice, AMNs are called lattice-based

B-splines

B-splines have been employed in surface-fitting algorithms for computer-aided design tasks [9] and they are recursively defined byNjk+1(x)=x−λjλj+k−1−λjNjk(x)+λj+k−xλj+k−λj+1Nj+1k(x),Nj1(x)=1ifx∈[λjj+1),0otherwisewith some valuable characteristics such as

  • Positivity:Njk⩾0forallx,

  • Local support:Njk=0ifx∉[λjj+k],

  • Partition of unity:j=1lNjk(x)=1,x∈[λjj+1].

B-spline functions are piecewise polynomial mappings formed from linear combinations of weighted basis functions and by using B-splines as

Evolutionary optimized LB-AMNs

The problem is to find a (sub)optimal LB-AMN by using only important inputs from a given input matrix and to build up a network topology with as few as possible free parameters which can handle the problem sufficiently. Each LB-AMN is defined by

  • the used inputs (columns) of the input matrix;

  • the order(s) of the basis functions that cover the input(s);

  • the number of basis functions used to cover the input(s);

  • the knotpositions which define the shape of the basis functions.

To find (sub)optimal values

A simple mathematical problem

To take a first look on the capability of our approach we optimize the topology of a B-spline LB-AMN for a one-dimensional mathematical function given byf(x)=10(e−5|x|+e−3|x−0.8|/10+e−10|x+0.6|).Using 201 training patterns in the interval [−1,1] and parameters as given below the (sub)optimal B-spline LB-AMN was calculated. A test error was not computed since the use of maximal 12 weights to model non-linear data with two discontinuities will retrench overfitting. This example should only

Real world toxicity data

Considering a difficult real world problem we examined toxicity data. The toxicity dataset provided by [14] consists of 211 pesticides from six different chemical classes with data on toxicity for Daphnia Magna (a water flea). Each molecule is described by about 160 descriptors which were used as system input and the desired output is the toxicity for Daphnia Magna (Fig. 8). For further information about the dataset see [15].

Each molecule of the dataset is classified up to three different

Conclusion

We presented an approach for problem specific structure identification optimization of LB-AMNs with the possibility to use nearly arbitrary basis functions. The algorithm performs well and for high-dimensional problems it is possible to detect relevant inputs. Furthermore the optimal number and shape of the basis functions can be found. More research will be done on accelerating the genetic algorithm by an “intelligent” control system, considering generalized basis functions and by constructing

Acknowledgements

This work has been supported by the Commission of the European Communities under the Program “Environment and Climate”, Project “COMET”, Contract No. ENV4-CT97-0508.

References (15)

  • D.S. Broomhead et al.

    Multivariable functional interpolation and adaptive networks

    Complex Systems

    (1988)
  • J.S. Albus

    A new approach to manipulator control: the cerebellar model articulation controller (CMAC)

    Trans. ASME Jnl. Dyn. Sys. Meas. and Control

    (1975)
  • J. Moody, Fast learning in multi-resolution hierachies, in: D.S. Touretzky (Ed.), Advances in Neural Information...
  • P. Kanerva

    Sparse Distributed Memory

    (1988)
  • P. Kanerva, Associative-memory models of the cerebellum, in: I. Aleksander, J. Tayler (Eds.), Artificial Neural...
  • R.S. Martin et al.

    Symetric decomposition of positive bandmatrices

    Numerische Mathematik

    (1965)
  • W.H. Press

    Numerical Recipes in C: The Art of Scientific Computing

    (1996)
There are more references available in the full text version of this article.

Cited by (2)

View full text