Evolving Features in Neural Networks for System Identification

Abstract

Given N data pairs {X _i , y _i }, i=1,2,...,N, where each X _i is an n-dimensional vector of independent variables $(X_i = <x_{i_1} , x_{i_2}, \ldots,x_{i_n}>)$ and y _i is a dependent variable, the function approximation problem (FAP) is finding a function that best explains the N pairs of X _i and y _i . From the universal approximation theorem and inherent approximation capabilities proved by various researches, artificial neural networks (ANNs) are considered as powerful function approximators. There are two main issues on the feedforward neural networks’ performance. One is to determine its structure. The other issue is to specify the weights of a network that minimizes its error. Genetic algorithm (GA) is a global search technique and is useful for complex optimization problems. So, it has been considered to have potential to reinforce the performance of neural networks. Many researchers tried to optimize the weights of networks using genetic algorithms alone or combined with the backpropagation algorithm. Others also tried to find a good topology that is even more difficult and called a “black art.”