Evolving RBF neural networks for time-series forecasting with EvRBF

Abstract

This paper is focused on determining the parameters of radial basis function neural networks (number of neurons, and their respective centers and radii) automatically. While this task is often done by hand, or based in hillclimbing methods which are highly dependent on initial values, in this work, evolutionary algorithms are used to automatically build a radial basis function neural networks (RBF NN) that solves a specified problem, in this case related to currency exchange rates forecasting. The evolutionary algorithm EvRBF has been implemented using the evolutionary computation framework evolving object, which allows direct evolution of problem solutions. Thus no internal representation is needed, and specific solution domain knowledge can be used to construct specific evolutionary operators, as well as cost or fitness functions. Results obtained are compared with existent bibliography, showing an improvement over the published methods.

Introduction

A radial basis function (RBF), φ, can be characterized by a point of the input space, c, and a radius or width, r, such that the RBF reaches its optimum value (maximun/minimun) when applied to c, and decreases/increases to its opposite optimum value when applied to points far from c. The radius, r, controls how distance affects that increment or decrement. For this reason, mathematicians have used groups of RBF to successfully interpolate data. Typical examples of RBF are the Gaussian function (see Fig. 1a) and the multiquadric function (see Fig. 1b), although there are many others [1], [2].

RBF are used by radial basis function neural networks (RBF NNs), which were introduced by Broomhead and Lowe in [3], being their main applications function approximation and time-series forecasting, as well as classification or clustering tasks. Traditionally, a RBF NN is thought as a two-layer, feed-forward network in which hidden neuron activation functions are RBF (see Fig. 2). Very often, the function used is the Gaussian one.

In RBF NNs each hidden neuron computes the distance from its input to the neuron's central point, c, and applies the RBF to that distance, as shows Eq. (1).

$$\text{h}{}_{\mathrm{i}}\text{(x)=φ}\text{∥x−c}{}_{\mathrm{i}}\text{∥}{}^2\text{/r}{}_{\mathrm{i}}{}^2$$

where h_i(x) is the output yielded by hidden neuron number i when input x is applied; φ is the RBF, c_i is the center of the ith hidden neuron, and r_i is its radius.

In the next step, the neurons of the output layer perform a weighted sum using the outputs of the hidden layer and the weights of the links that connect both output and hidden layer neurons (Eq. (2)).

$$\text{o}{}_{\mathrm{j}}\text{(x)=∑}{}_{\mathrm{i=0}}{}^{\mathrm{n-1}}\text{w}{}_{\mathrm{ij}}\text{h}{}_{\mathrm{i}}\text{(x)+w}{}_{\mathrm{0j}}$$

where o_j(x) is the value yielded by output neuron number j when input x is applied; w_ij is the weight of the links that connects hidden neuron number i and output neuron number j, w_0j is a bias for the output neuron, and finally, n is the number of hidden neurons.

It has been proved [4] that when enough units are provided, a RBF NN can approximate any multivariate continuous function as much as desired. This is possible because once the centers and the radii have been fixed, the weights of the links between the hidden and the outputs layers can be calculated analytically using singular value decomposition [5] or any algorithm suitable to solve lineal algebraic equations, making unnecessary the use of training algorithms, as those used in other kinds of neural networks such as multilayer perceptrons. Thus, the main problem in RBF NNs design concerns establishing the number of hidden neurons to use and their centers and radii.

The need of automatic mechanisms to build RBF NNs is already present in Broomhead and Lowe's work [3], where they showed that one of the parameters that critically affects the performance of RBF NNs is the number of hidden neurons. When this number is not sufficient, the approximation offered by the net is not good enough; in the other hand, nets with many hidden neurons will approximate very well those points used to calculate the connection weights, while having a very poor predictive power; this is the so-called overfitting problem [6]. In consequence, establishing the number of neurons (that is, the number of centers and values related to them) that solves a given problem is one of the most important tasks researchers have faced in this field.

In this paper an evolutionary algorithm is used to find the best components of the RBF NNs that approximate a function representing a time-series. Results are compared with other techniques that also use the evolutionary approach to tune the RBF NN.

The rest of the paper is organized as follows: Section 2 describes some of the methods used to solve the cited problems; Section 3 shows our proposed solution, and describes the evolutionary computation framework we use (EO). Next section (4) shows some functional approximation experiments; and finally, our conclusions and future working lines are exposed in Section 5.