Abstract
Radial basis function (RBF) networks of Gaussian activation functions have been widely used in many applications due to its simplicity, robustness, good approximation and generalization ability, etc.. However, the training of such a RBF network is still a rather difficult task in the general case and the main crucial problem is how to select the number and locations of the hidden units appropriately. In this paper, we utilize a new kind of Bayesian Ying-Yang (BYY) automated model selection (AMS) learning algorithm to select the appropriate number and initial locations of the hidden units or Gaussians automatically for an input data set. It is demonstrated well by the experiments that this BYY-AMS training method is quite efficient and considerably outperforms the typical existing training methods on the training of RBF networks for both clustering analysis and nonlinear time series prediction.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Broomhead, D.S., Lowe, D.: Multivariable Functional Interpolation and Adaptive Networks. Complex System 2, 321–355 (1988)
Moody, J., Darken, C.: Fast Learning in Networks of Locally Tuned Processing Units. Neural Computation 1, 281–294 (1989)
Poggio, T., Girosi, F.: Regularization Algorithms for Learning that are Equivalent to Multiplayer Networks. Science 247, 978–982 (1990)
Platt, J.: A Resource-allocating Network for Function Interpolation. Neural Computation 3, 213–225 (1991)
Esposito, A., Marinaro, M., Oricchio, D., Scarpetta, S.: Approximation of Continuous and Discontinuous Mapping by a Growing Neural RBF-based Algorithm. Neural Networks 13, 651–665 (2000)
Sanchez, A.V.D.: Searching for a Solution to the Automatic RBF Network Design Problem. Neurocomputing 42, 147–170 (2002)
Hartigan, J.A.: Distribution Problems in Clustering. In: Van Ryzin, J. (ed.) Classification and clustering, pp. 45–72. Academic Press, New York (1977)
Xu, L., Krzyzak, A., Oja, E.: Rival Penalized Competitive Learning for Clustering Analysis, Rbf Net and Curve Detection. IEEE Trans. on Neural Networks 4, 636–649 (1993)
Ma, J., Wang, T.: A Cost-functionApproach to Rival Penalized Competitive Learning (RPCL). IEEE Transactions on Systems, Man and Cybernetics, Part B: Cybernetics 36(4), 722–737 (2006)
Krzyzak, A., Linder, T., Lugosi, G.: Nonparametric Estimation and Classification Using Radial Basis Function Nets and Empirical Risk Minimization. IEEE Trans. on Neural Networks 7, 475–487 (1996)
Bors, A.G., Pitas, I.: Median radial basis function neural network. IEEE Transactions on Neural Networks 7, 1351–1364 (1996)
Sohn, I., Ansari, N.: Configure Rbf Neural Networks. Electronics Letters 34, 684–685 (1998)
Beni, G., Liu, X.: A Least Biased Fuzzy Clustering Method. IEEE Transactions On Pattern Analysis and Machine Intelligence 16, 954–960 (1994)
Xu, L.: Ying-Yang Machine: A Bayesian Kullback Scheme for Unified Learning and New Results on Vector Quantization. In: Proceedings of International Conference on Neural Information Processing (ICONIP95), vol. 2, pp. 977–988 (1995)
Xu, L.: A Unified Learning Scheme: Bayesian-Kullback Ying-Yang Machine. Advances in Neural Information Processing Systems 8, 444–450 (1996)
Xu, L.: Best Harmony, Unified RPCL and Automated Model Selection for Unsupervised and Supervised Learning on Gaussian mixtures,Three-layer Nets and Me-rbf-svm Model. International Journal of Neural System 11, 43–69 (2001)
Ma, J., Wang, T., Xu, L.: A Gradient BYY Harmony Learning Rule on Gaussian Mixture with Automated Model Selection. Neurocomputing 56, 481–487 (2004)
Ma, J., Gao, B., Wang, Y., Cheng, Q.: Conjugate and Natural Gradient Rules for BYY Harmony Learning on Gaussian Mixture with Automated Model Selection. International Journal of Pattern Recognition and Artificial Intelligence 19, 701–713 (2005)
Ma, J., Wang, L.: BYY Harmony Learning on Finite Mixture: Adaptive Gradient Implementation and a Floating RPCL Mechanism. Neural Processing Letters 24, 19–40 (2006)
Robbins, H., Monro, S.: A Stochastic Approximation Method. Annals of Mathematical Statistics 22, 400–407 (1951)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, K., Wang, L., Ma, J. (2007). Efficient Training of RBF Networks Via the BYY Automated Model Selection Learning Algorithms. In: Liu, D., Fei, S., Hou, ZG., Zhang, H., Sun, C. (eds) Advances in Neural Networks – ISNN 2007. ISNN 2007. Lecture Notes in Computer Science, vol 4491. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72383-7_138
Download citation
DOI: https://doi.org/10.1007/978-3-540-72383-7_138
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72382-0
Online ISBN: 978-3-540-72383-7
eBook Packages: Computer ScienceComputer Science (R0)