CONTRIBUTED ARTICLEFunction Emulation Using Radial Basis Function Networks
Section snippets
INTRODUCTION
When humans learn a task, it is natural to first get at the qualitative structure of the task, with accuracy improving with practice. For instance, in the initial stages of learning to perform a complex motor task, the amateur is guided by some qualitative rules or “tips” from the master. Once skill improves by practice and feedback, the amateur-turned-expert need not follow the rules consciously any longer. But it is perhaps impossible for humans to learn most tasks solely by imitating the
Critical Points
Catastrophe theory (CT) is mainly concerned with systems whose dynamics can be described by a smooth1 potential f(x), where x = (x1,… xn) is the vector of state variables, f(x;c) is a family of potentials, c = (c1,…, ck) is the vector of control parameters, and “▽x” is the gradient operator. Such systems are known as gradient systems. Qualitative properties of the system (stable? unstable? how many minima?) can be understood
FUNCTION EMULATION USING A FEEDFORWARD NEURAL NETWORK
We may now formulate the concept of function emulation, whereby the network aims to learn the qualitative features of the target function. We say that a network f(x) emulates a target function t(x) if all of the following conditions are satisfied.
- 1.
(i) f(x) and t(x) have the same number of CPs.
- 2.
(ii) Let ξ1,… ξm be CPs of the target t(x). There exists an ordering ζ1,…, ζm of the CPs of f(x), such that: t(x)|x=ξi≐f(x)|x=ζi for i = 1,…, m.
- 3.
(iii) Construct two Voronoi tessellations of the input space Rn
NUMERICAL STUDIES
In this section, we apply the training method of the previous section to emulate functions with a single CP. The number of nodes required for emulating NDCPs and DCPs of type Ak+1 is determined. For these cases, the network size is shown to be related to codimension. Preliminary experiments suggesting a similar relation for higher catastrophes (Dk+1, E6 etc.) are also conducted. A method for finding the desired network size when the type of target CP is unknown is also described.
To emulate a
EMULATING TARGETS WITH MULTIPLE CRITICAL POINTS
In general a smooth target function has multiple CPs. For emulating such targets we prescribe a modular architecture in which each module emulates the target in the neighborhood of a CP. The entire scheme has four components:
- 1.
1. Find CPs of target function from training data.
- 2.
2. Partition training data Q as Q = Q1 ⋃ Q2… ⋃ Qn, and Qr ⋂ Qs = φ for r ≠ s.
- 3.
3. Train module Ms with Qs.
- 4.
4. Combine modules.
The above subtasks are now described.
- 1.
(i) Finding CPs of target function from training data: It is
DISCUSSION
Under certain conditions, catastrophe theory provides a classification of critical points (CPs) of smooth multivariate functions and hence a basis for function emulation. The present training procedure, designed to achieve this goal using neural networks, is based on the principle that local qualitative structure of a smooth function depends on the early terms of its Taylor series expansion around the CPs. The Taylor series can be truncated at a certain point without disturbing the qualitative
Acknowledgements
The work described here was supported in part by the National Science Foundation under grant ECS-9307632 and by AFOSR Contract F49620-93-1-0307.
References (17)
- et al.
Multivariable functional interpolation and adaptive networks
Complex Systems
(1988) - et al.
Scale-based clustering using the Radial Basis Function network
IEEE Transactions on Neural Networks
(1996) - Friedman, J. (1994). An overview of predictive learning and function approximation. In V. Cherkassky, J.F. and H....
- Gilmore, R. (1981). Catastrophe theory for scientists and engineers. New York:...
- Golub, G., & Van Loan, C. (1989). Matrix computations. Baltimore, MD: Johns Hopkins University...
- Hoskins, J.C., Lee, P., & Chakravarthy, S.V. (1993). Polynomial modeling behavior in radial basis function networks. In...
- Kadirkamanathan, V., Niranjan, M., & Fallside, F. (1991). Sequential adaptation of radial basis function neural...
- et al.
Multiple representations of space underlying behavior
Behavioral Brain Sciences
(1982)
Cited by (5)
The shape of handwritten characters
2003, Pattern Recognition LettersApproximation of function and its derivatives using radial basis function networks
2003, Applied Mathematical ModellingMesh-free radial basis function network methods with domain decomposition for approximation of functions and numerical solution of Poisson's equations
2002, Engineering Analysis with Boundary ElementsThe unreasonable effectiveness of neural network approximation
1999, IEEE Transactions on Automatic ControlPublic transit passenger transportation quantity forecast based on synthetic integrated multi-variable grey model
2009, Proceedings - International Conference on Management and Service Science, MASS 2009