Abstract
The well-known mixtures of experts(ME) model is usually trained by expectation maximization(EM) algorithm for maximum likelihood learning. However, we have to first determine the number of experts, which is often hardly known. Derived from regularization theory, a regularized minimum cross-entropy(RMCE) algorithm is proposed to train ME model, which can automatically make model selection. When time series is modeled by ME, it is demonstrated by some climate prediction experiments that RMCE algorithm outperforms EM algorithm. We also compare RMCE algorithm with other regression methods such as back-propagation(BP) algorithm and normalized radial basis function(NRBF) network, and find that RMCE algorithm still shows promising results.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Lu, Z., Cheng, Q., Ma, J.: A Gradient BYY Harmony Learning Algorithm on Mixture of Experts for Curve Detection. In: Gallagher, M., Hogan, J.P., Maire, F. (eds.) IDEAL 2005. LNCS, vol. 3578, pp. 250–257. Springer, Heidelberg (2005)
Weigend, A.S., Manageas, M., Srivastava, A.N.: Nonlinear Gated Experts for Time Series: Discovering Regimes and Avoiding Over-fitting. Int. J. Neural Systems 6, 373–399 (1995)
Jordan, M.I., Xu, L.: Convergence Results for the EM Approach to Mixtures-of-Experts Architectures. Neural Networks 8, 1409–1431 (1995)
Pal, N.R.: On Minimum Cross-Entropy Thresholding. Pattern Recognition 29, 575–580 (1996)
Vapnik, V.N.: An Overview of Statistical Learning Theory. IEEE Trans. Neural Networks 10, 988–999 (1999)
Lee, T.L.: Back-Propagation Neural Network for Long-Term Tidal Predictions. Ocean Engineering 31, 225–238 (2004)
Moody, J., Darken, J.: Fast Learning in Networks of Locally-Tuned Processing Units. Neural Comput. 1, 281–294 (1989)
Xu, L.: BYY Harmony Learning, Structural RPCL, and Topological Self-Organizing on Mixture Modes. Neural Networks 15, 1231–1237 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Lu, Z. (2006). A Regularized Minimum Cross-Entropy Algorithm on Mixtures of Experts for Time Series Prediction. In: Wang, J., Yi, Z., Zurada, J.M., Lu, BL., Yin, H. (eds) Advances in Neural Networks - ISNN 2006. ISNN 2006. Lecture Notes in Computer Science, vol 3972. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11760023_111
Download citation
DOI: https://doi.org/10.1007/11760023_111
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-34437-7
Online ISBN: 978-3-540-34438-4
eBook Packages: Computer ScienceComputer Science (R0)