Normalized Gaussian Radial Basis Function networks

doi:10.1016/S0925-2312(98)00027-7

Neurocomputing

Volume 20, Issues 1–3, 31 August 1998, Pages 97-110

https://doi.org/10.1016/S0925-2312(98)00027-7 Get rights and content

Abstract

The performances of normalised RBF (NRBF) nets and standard RBF nets are compared in simple classification and mapping problems. In normalized RBF networks, the traditional roles of weights and activities in the hidden layer are switched. Hidden nodes perform a function similar to a Voronoi tessellation of the input space, and the output weights become the network's output over the partition defined by the hidden nodes. Consequently, NRBF nets lose the localized characteristics of standard RBF nets and exhibit excellent generalization properties, to the extent that hidden nodes need to be recruited only for training data at the boundaries of class domains. Reflecting this, a new learning rule is proposed that greatly reduces the number of hidden nodes needed in classification tasks. As for mapping applications, it is shown that NRBF nets may outperform standard RBFs nets and exhibit more uniform errors. In both applications, the width of basis functions is uncritical, which makes NRBF nets easy to use.

Introduction

Normalized radial basis functions (NRBF) differ from standard radial basis functions (RBF) by a seemingly minor modification of their equation (Section 2). This results in novel computational properties which have attracted little attention in the neural network community. Moody and Darken [11]were first to mention normalised RBF nets without elaborating on their functional significance. However, Servin and Cuevas (1993) noted that normalization gave RBF nets the “same classification properties as nets using sigmoid functions”. Cha and Kassam (1995) proposed that “a normalized Gaussian basis function features either localized behaviour similar to that of a Gaussian or nonlocalized behavior like that of a sigmoid, depending on the location of its centre”. Rao et al. [13]interpreted NRBF nets as mixture of expert models and Jang and Sun [7]saw similarities with fuzzy inference systems. These multiple views reflect the fact that NRBF nodes in the hidden layer behave more like case indicators rather than basis functions proper, as is elaborated in Section 2. This property leads to excellent performances in classification tasks, as shown in Section 3. One of the key features of NRBF nets is their excellent generalization, a property that can be exploited to reduce the number of hidden nodes in classification tasks. This is achieved by using a new learning rule proposed in Section 4that is demonstrated in classification and mapping examples. NRBF nets have also given very good results in another class of application, trajectory learning in robotics 1, 6.

Section snippets

Normalized radial basis function networks

Standard radial basis function (RBF) nets (Fig. 1) comprise a hidden layer of RBF nodes and an output layer with linear nodes 4, 11The function of these nets is given by $y_{i} (x)= ∑ j=1 n w_{ij} φ(x−x_{j}),$ where y_i is the activity of the output node i, φ(x−x_j) is the activity of the hidden node j, with a RBF function centred on the vector x_j, x is the actual input vector and w_ij are the weights from the RBF nodes in the hidden layer to the linear output node. Such a net is a universal function approximator

Standard training procedure

For the results shown in this section, training is done in a standard way [3], (p. 170), by recruiting hidden nodes in the first epoch, then, in subsequent epochs, adjusting the positions of the centres of the nodes and the weights to the output node to minimize the output error. Normalised RBF nets and standard RBF nets are trained with the same procedure:

(i) Recruiting a new hidden node centred on an input vector that was beyond a radius of 0.5σ from the centre of an existing node, or slowly

Modified training procedure

The good interpolation and extrapolation properties shown in the previous section suggests that hidden nodes may need to be recruited only in crucial points, close to boundaries between two classes. To verify this hypothesis, the training procedure in Section 3.1was modified so that no new nodes are recruited if the network indicates the correct class by using existing nodes. The details of the modified procedure are:

(i) Check if the output vector y of the net is correct, i.e. |y−y_desired|<

Conclusion

These initial results show that normalised RBF nets have very good generalisation properties that are beneficial in classification tasks. This is due to the property of normalised RBF nets to produce a significant output even for input vectors far from the centre of the receptive field of any of the hidden nodes.

Taking advantage of this, a modified learning procedure has been proposed in which hidden nodes are recruited only when neighbouring nodes do not point to the same output value. The

Acknowledgements

Stimulating comments and suggestions by Kaspar Althoefer and anonymous referees are gratefully acknowledged and have helped to improve the final version of this paper.

References (14)

K. Althoefer, G. Bugmann, Planning and learning goal-directed sequences of robot-arm movements, in: F. Fogelman-Soulié,...
H.C. Andersen, A. Lotfi, L. Westphal, Comments on functional equivalence between radial basis function networks and...
C.M Bishop
Neural Networks for Pattern Recognition
(1995)
D.S Broomhead et al.
Multivariable functional interpolation and adaptive networks
Complex Systems
(1988)
M Brown et al.
Neurofuzzy Adaptive Modelling and Control
(1994)
G. Bugmann, K.L. Koay, N. Barlow, M. Phillips, D. Rodney, Stable encoding of robot trajectories using normalised radial...
J.-S.R. Jang, C.-T. Sun, Functional equivalence between radial basis function networks and fuzzy inference systems,...

There are more references available in the full text version of this article.

Cited by (132)

Reconstruction of measurement data with multiple outliers using novel domain-based RBF
2024, Mechanical Systems and Signal Processing
Due to the high accuracy of computation, radial basis function (RBF) is widely recognized as a versatile and effective method for interpolating and approximating discrete points in various fields. However, RBF is quite sensitive to outliers, which can easily lead to distorted results. In this article, a novel overlapped domain-based RBF (ODRBF) method is proposed, in which the concept of effective domain is introduced to build a moving model, and Student’s t-regression and Gaussian mixture model (GMM) clustering are used for dealing with local anomalies. By introducing the effective domain, the estimated points and domain radius are constructed and the global model can be transformed into local estimation models. In each effective domain, a series of estimation models are iteratively generated through Student’s t-regression, and based on the distances between the estimation model and discrete points, GMM clustering is used to subsequently select the data as the input of the next regression. This iterative strategy in each effective domain ensures the removal of multiple outliers. Then, the preserved points in the processed effective domain are used to obtain local estimated value by RBF. The proposed method demonstrates strong robustness to highly contaminated dataset in the reconstruction of the simulation and experimental datasets.
Parameter control for differential evolution by storage of successful values at an individual level
2023, Journal of Computational Science
Setting the parameters of differential evolution to appropriate values is critical for attaining satisfactory performance on a given problem (class). Therefore, parameter tuning is typically necessary, at least to some extent. However, even despite extensive tuning, the best possible performance may still not be achieved since different parameter values might be necessary during the different phases of the optimisation process. A solution to this issue is offered by parameter control schemes that properly adjust the parameters during the algorithm run. This paper presents a new self-adaptive parameter control scheme, which is based on the storage of a few previously successful values for each population member separately. These are then subsequently used to generate new and better-fitting parameter values with the aim of maintaining an effective search throughout the algorithm run. The highly competitive results obtained on challenging benchmark functions of different dimensionality suggest its viability and robustness. This is further corroborated by the promising results obtained on the problem of automatic radial basis function network design for classification needs.
Response surface methodology for performance evaluation of insensitive munitions wastewater membrane filtration
2023, Cleaner Engineering and Technology
Munitions in industrial wastewater is an emerging area of concern with few efficient and economical treatment options. Membrane separation technologies could minimize munitions wastewater volumes, but have not been fully investigated. Herein, a response surface model (RSM) is used to investigate flux and munition rejections from simulated wastewater as a function of pH, transmembrane pressure, and co- and counterion concentrations. Rejection of 3-nitro-1,2,4-triazol-5-one (NTO), hexahydro-1,3,5-trinitro-1,3,5-triazine (RDX), 2,4-dinitroanisole (DNAN) and nitroguanidine (NQ) is determined for reverse osmosis (RO) membranes, namely BW30 and SW30XLE. Rejection using reverse osmosis is between 97.5% and 99.9% for NTO, 89.9%–99.4% for RDX, 96.1%–98.8% for DNAN, and 80.4%–96.2% for NQ. The RO membrane rejection is stable over 4 or more cycles with munitions spiked domestic wastewater and flux is rapidly regained after regeneration with 0.05% NaOH. Flux and performance index are twice as high for the BW30 membrane compared to the SW30XLE membrane. Operational costs for RO of munitions wastewater are 0.138 USD/m³ assuming 0.07 USD/kWh and fluxes of 0.68 ± 0.21 m³/m²/day. Gaussian process models are preferred for modeling the effect of pH, ionic concentration and pressure on membrane performance. Results herein suggest that reverse osmosis should be considered in munitions wastewater treatment trains.
Differential radial basis function network for sequence modelling[Formula presented]
2022, Expert Systems with Applications
We propose a differential radial basis function (RBF) network termed RBF-DiffNet—whose hidden layer blocks are partial differential equations (PDEs) linear in terms of the RBF—to make the baseline RBF network robust to noise in sequential data. Assuming that the sequential data derives from the discretisation of the solution to an underlying PDE, the differential RBF network learns constant linear coefficients of the PDE, consequently regularising the RBF network by following modified backward-Euler updates. We experimentally validate the differential RBF network on the logistic map chaotic timeseries as well as on 30 real-world timeseries provided by Walmart in the M5 forecasting competition. The proposed model is compared with the normalised and unnormalised RBF networks, ARIMA, and ensembles of multilayer perceptrons (MLPs) and recurrent networks with long short-term memory (LSTM) blocks. From the experimental results, RBF-DiffNet consistently shows a marked reduction in the prediction error over the baseline RBF network (e.g., 41% reduction in the root mean squared scaled error on the M5 dataset, and 53% reduction in the mean absolute error on the logistic map); RBF-DiffNet also shows a comparable performance to the LSTM ensemble but requires 99% less computational time. Our proposed network consequently enables more accurate predictions—in the presence of observational noise—in sequence modelling tasks such as timeseries forecasting that leverage the model interpretability, fast training, and function approximation properties of the RBF network.
Computational reproductions of external force field adaption without assuming desired trajectories
2021, Neural Networks
Optimal feedback control is an established framework that is used to characterize human movement. However, it is not fully understood how the brain computes optimal gains through interactions with the environment. In the past study, we proposed a model of motor learning that identifies a set of feedback and feedforward controllers and a state predictor of the arm musculoskeletal system to control free reaching movements. In this study, we applied the model to force field adaptation tasks where normal reaching movements are disturbed by an external force imposed on the hand. Without a priori knowledge about the arm and environment, the model was able to adapt to the force field by generating counteracting forces to overcome it in a manner similar to what is reported in the behavioral literature. The kinematics of the movements generated by our model share characteristic features of human movements observed before and after force field adaptation. In addition, we demonstrate that the structure and learning algorithm introduced in our model induced a shift in the end-point’s equilibrium position and a static force modulation, accompanied by a fast and a slow learning process. Importantly, our model does not require desired trajectories, yields movements without specifying movement duration, and predicts force generation patterns by exploring the environment. Our model demonstrates a possible mechanism through which the central nervous system may control and adapt a point-to-point reaching movement without specifying a desired trajectory by continuously updating the body’s musculoskeletal model.
A joint multi-model machine learning prediction approach based on confidence for ship stability
2024, Complex and Intelligent Systems

View all citing articles on Scopus

Guido Bugmann was born in 1953 and has two children. He studied Physics at the University of Geneva in Switzerland. In 1986 he completed his PhD on “Fabrication of photovoltaic solar cells with a-Si : H produced by anodic deposition in a DC plasma”. He then worked at the Swiss Federal Institute of Technology in Lausanne on the development of a measurement system using an ultra-sound beam and neural networks to measure the size of air bubbles in bacterial cultures. In 1989, he joined the Fundamental Research Laboratories of NEC in Japan and modelled the function of biological neurons in the visual system. In 1992 he joined Prof. John G. Taylor at King's College London to develop applications of the pRAM neuron model and develop a theory of visual latencies. In 1993 he joined the group of Prof. Mike Denham at the University of Plymouth (UK) where he is developing artificial vision systems for robots and investigates path-planning and spatial memory.

Dr. Bugmann has 3 patents and over 70 publications. He is member of the Swiss Physical Society, The Neuroscience Society and The British Machine Vision Association.

View full text

Normalized Gaussian Radial Basis Function networks

Abstract

Introduction

Section snippets

Normalized radial basis function networks

Standard training procedure

Modified training procedure

Conclusion

Acknowledgements

Neural Networks for Pattern Recognition

Multivariable functional interpolation and adaptive networks

Complex Systems

Neurofuzzy Adaptive Modelling and Control