Elsevier

Neural Networks

Volume 10, Issue 3, April 1997, Pages 459-478
Neural Networks

CONTRIBUTED ARTICLE
Function Emulation Using Radial Basis Function Networks

https://doi.org/10.1016/S0893-6080(96)00091-3Get rights and content

Abstract

While learning an unknown input-output task, humans first strive to understand the qualitative structure of the function. Accuracy of performance is then improved with practice. In contrast, existing neural network function approximators do not have an explicit means for abstracting the qualitative structure of a target function. To fill this gap, we introduce the concept of function emulation, according to which the central goal of training is to “emulate” the qualitative structure of the target function. The framework of catastrophe or singularity theory is used to characterize the qualitative structure of a smooth function, which is organized by the critical points of the function. The proposed scheme of function emulation uses the radial basis function network to realize a modular architecture wherein each module emulates the target function in the neighborhood of a critical point. The network size required to emulate the target in the neighborhood of a critical point is shown to be related to a certain complexity measure of the critical point. For a large class of smooth functions, the present scheme produces a graph-like abstraction of the target, thereby providing a qualitative representation of a quantitative input-output relation. © 1997 Elsevier Science Ltd. All Rights Reserved.

Section snippets

INTRODUCTION

When humans learn a task, it is natural to first get at the qualitative structure of the task, with accuracy improving with practice. For instance, in the initial stages of learning to perform a complex motor task, the amateur is guided by some qualitative rules or “tips” from the master. Once skill improves by practice and feedback, the amateur-turned-expert need not follow the rules consciously any longer. But it is perhaps impossible for humans to learn most tasks solely by imitating the

Critical Points

Catastrophe theory (CT) is mainly concerned with systems whose dynamics can be described by a smooth1 potential f(x),dxdt=−∇xf(x;c), where x = (x1,… xn) is the vector of state variables, f(x;c) is a family of potentials, c = (c1,…, ck) is the vector of control parameters, and “▽x” is the gradient operator. Such systems are known as gradient systems. Qualitative properties of the system (stable? unstable? how many minima?) can be understood

FUNCTION EMULATION USING A FEEDFORWARD NEURAL NETWORK

We may now formulate the concept of function emulation, whereby the network aims to learn the qualitative features of the target function. We say that a network f(x) emulates a target function t(x) if all of the following conditions are satisfied.

  • 1.

    (i) f(x) and t(x) have the same number of CPs.

  • 2.

    (ii) Let ξ1,… ξm be CPs of the target t(x). There exists an ordering ζ1,…, ζm of the CPs of f(x), such that: t(x)|x=ξif(x)|x=ζi for i = 1,…, m.

  • 3.

    (iii) Construct two Voronoi tessellations of the input space Rn

NUMERICAL STUDIES

In this section, we apply the training method of the previous section to emulate functions with a single CP. The number of nodes required for emulating NDCPs and DCPs of type Ak+1 is determined. For these cases, the network size is shown to be related to codimension. Preliminary experiments suggesting a similar relation for higher catastrophes (Dk+1, E6 etc.) are also conducted. A method for finding the desired network size when the type of target CP is unknown is also described.

To emulate a

EMULATING TARGETS WITH MULTIPLE CRITICAL POINTS

In general a smooth target function has multiple CPs. For emulating such targets we prescribe a modular architecture in which each module emulates the target in the neighborhood of a CP. The entire scheme has four components:

  • 1.

    1. Find CPs of target function from training data.

  • 2.

    2. Partition training data Q as Q = Q1Q2… ⋃ Qn, and QrQs = φ for rs.

  • 3.

    3. Train module Ms with Qs.

  • 4.

    4. Combine modules.

The above subtasks are now described.

  • 1.

    (i) Finding CPs of target function from training data: It is

DISCUSSION

Under certain conditions, catastrophe theory provides a classification of critical points (CPs) of smooth multivariate functions and hence a basis for function emulation. The present training procedure, designed to achieve this goal using neural networks, is based on the principle that local qualitative structure of a smooth function depends on the early terms of its Taylor series expansion around the CPs. The Taylor series can be truncated at a certain point without disturbing the qualitative

Acknowledgements

The work described here was supported in part by the National Science Foundation under grant ECS-9307632 and by AFOSR Contract F49620-93-1-0307.

References (17)

  • D. Broomhead et al.

    Multivariable functional interpolation and adaptive networks

    Complex Systems

    (1988)
  • S.V. Chakravarthy et al.

    Scale-based clustering using the Radial Basis Function network

    IEEE Transactions on Neural Networks

    (1996)
  • Friedman, J. (1994). An overview of predictive learning and function approximation. In V. Cherkassky, J.F. and H....
  • Gilmore, R. (1981). Catastrophe theory for scientists and engineers. New York:...
  • Golub, G., & Van Loan, C. (1989). Matrix computations. Baltimore, MD: Johns Hopkins University...
  • Hoskins, J.C., Lee, P., & Chakravarthy, S.V. (1993). Polynomial modeling behavior in radial basis function networks. In...
  • Kadirkamanathan, V., Niranjan, M., & Fallside, F. (1991). Sequential adaptation of radial basis function neural...
  • I. Lieblich et al.

    Multiple representations of space underlying behavior

    Behavioral Brain Sciences

    (1982)
There are more references available in the full text version of this article.

Cited by (5)

  • The shape of handwritten characters

    2003, Pattern Recognition Letters
  • The unreasonable effectiveness of neural network approximation

    1999, IEEE Transactions on Automatic Control
  • Public transit passenger transportation quantity forecast based on synthetic integrated multi-variable grey model

    2009, Proceedings - International Conference on Management and Service Science, MASS 2009
View full text