Abstract
The Mixture of Experts model (ME) is a type of modular artificial neural network (MANN) whose architecture is composed by different kinds of networks who compete to learn different aspects of the problem. This model is used when the searching space is stratified. The learning algorithm of the ME model consists in estimating the network parameters to achieve a desired performance. To estimate the parameters, some distributional assumptions are made, so the learning algorithm and, consequently, the parameters obtained depends on the distribution. But when the data is exposed to outliers the assumption is not longer valid, the model is affected and is very sensible to the data as it is showed in this work. We propose a robust learning estimator by means of the generalization of the maximum likelihood estimator called M-estimator. Finally a simulation study is shown, where the robust estimator presents a better performance than the maximum likelihood estimator (MLE).
This work was supported in part by Research Grant Fondecyt 1010101 and 7010101, in part by ResearchGran t CHL-99/023 from the German Ministry of Education and Research( BMBF) and in part by Research Grant DGIP-UTFSM
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Allende, H., Moraga, C., Salas, R.: Neural model identification using local robustness analysis. In: Reusch, B. (ed.) Fuzzy Days 2001. LNCS, vol. 2206, pp. 162–173. Springer, Heidelberg (2001)
Allende, H., Moraga, C., Salas, R.: Robust estimator for the learning process in neural networks applied in time series. In: Dorronsoro, J.R. (ed.) ICANN 2002. LNCS, vol. 2415, pp. 1080–1086. Springer, Heidelberg (2002)
Dubois, D., Prade, H.: A review of fuzzy set aggregation connectives. Information Sciences 36, 85–121 (1985)
Hampel, F.R., Ronchetti, E.M., Rousseeuw, P.J., Stahel, W.A.: Robust Statistics. Wiley Series in Probability and Mathematical Statistics (1986)
Jacobs, R.A.: Computational studies of the development of functionally specialized neural modules. Trends in Cognitive Science 3(1), 31–38 (1999)
Jacobs, R.A., Jordan, M.I., Nowlan, S.J., Hinton, G.E.: Adaptive mixtures of local experts. Neural Computation 3(1), 79–87 (1991)
Jordan, M., Jacobs, R.: Modular and hierarchical learning systems, The Handbook of Brain Theory and Neural Networks, Cambridge, MA, vol. 1. MIT Press, Cambridge (1999)
Jordan, M.I., Jacobs, R.A.: Hierarchical mixtures of experts and the EM algorithm. Neural Computation 6(2), 181–214 (1994)
Moerland, P.: Mixtures of experts estimate a posteriori probabilities. In: Gerstner, W., Hasler, M., Germond, A., Nicoud, J.-D. (eds.) ICANN 1997. LNCS, vol. 1327, pp. 499–504. Springer, Heidelberg (1997)
Prechelt, L.: Proben1 – a set of benchmarks and benchmarking rules for neural training algorithms. Technical Report 21/94, Fakultaet fur Informatik, Universitaet Karlsruhe, D-76128 Karlsruhe, Germany (September 1994)
Ramamurti, V., Ghosh, J.: Structurally adaptive localized mixtures of experts for non-stationary enviroments. Technical report, University of Texas at Austin, Austin, TX 78712-1084 (1996)
Tadjudin, S., Landgrebe, D.: Robust parameter estimation for mixture model. IEEE Trans. on Geoscience and remote sensing 8(1), 439–445 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Allende, H., Torres, R., Salas, R., Moraga, C. (2003). Robust Learning Algorithm for the Mixture of Experts. In: Perales, F.J., Campilho, A.J.C., de la Blanca, N.P., Sanfeliu, A. (eds) Pattern Recognition and Image Analysis. IbPRIA 2003. Lecture Notes in Computer Science, vol 2652. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-44871-6_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-44871-6_3
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40217-6
Online ISBN: 978-3-540-44871-6
eBook Packages: Springer Book Archive