Abstract
In this paper an information geometric–based variable selection method for MLP networks is shown. It is based on divergence projections of the Riemannian manifold defined by a MLP network on submanifolds defined by MLP networks with reduced input dimension. We show how we can take advantage of the layered structure of the MLP to simplify the projection operation, which cannot be accurately done by using only the Fisher information metric. Furthermore, we show that our selection algorithm is more robust and gives better results than other well known selection algorithms like Optimal Brain Surgeon. Some examples are shown to validate the proposed approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Amari, S.: Information geometry of EM and em algorithms for neural networks. Neural Networks 8, 1379–1408 (1995)
Amari, S.: Methods of Information Geometry. Translations of Mathematical Monographs 191. Oxford University Press, Oxford (2000)
Dupuis, J.A., Robert, C.P.: Bayesian Variable Selection in Qualitative Models by Kullback–Leibler projections. J. Statistical Planning and Inference 111, 77–94 (2003)
Fukunaga, K.: Introduction to Statistical Pattern Recognition, 2nd edn. Academic Press, London (1990)
Hassibi, B., Stork, D.G.: Second order derivatives for network pruning: Optimal Brain Surgeon. In: Hanson, S.J., Cowanadn, J.D., Giles, C.L. (eds.) Advances in Neural Information Processing Systems 5: Proceedings of the 1992 Conference, pp. 164–171. Morgan Kaufmann, San Mateo (1993)
van de Laar, P., Heskes, T., Gielen, S.: Partial retraining: A new approach to input relevance determination. International Journal of Neural Systems A 9, 75–85 (1999)
LeCun, Y., Denker, J.S., Solla, S.A.: Optimal Brain Damage. In: Touretzky, D.S. (ed.) Advances in Neural Information Processing Systems 2: Proceedings of the 1989 Conference, pp. 598–605. Morgan Kaufmann, San Mateo (1990)
Leray, P., Gallinari, P.: Feature Selection with Neural Networks. Behaviormetrika (special issue on Analysis of Knowledge Representation in Neural Network Models) 26(1), 145–166 (1999)
MacKay, D.J.C.: A practical Bayesian framework for backpropagation networks. Neural Computation 4(3), 448–472 (1992)
McCullagh, P., Nelder, J.: Generalized Linear Models. Chapman and Hall, London (1989)
Neal, R.M.: Bayesian Learning for Neural Networks. Springer, New York (1996)
Stahlberger, A., Riedmiller, M.: Fast network pruning and feature extraction using the unit–OBS algorithm. In: Mozer, M., Jordan, M., Petsche, T. (eds.) Advances in Neural Information Processing Systems 9, pp. 655–661. MIT Press, Cambridge (1997)
Robert, C.P., Casella, G.: Markov Chain Monte Carlo Methods. Springer, New York (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Eleuteri, A., Tagliaferri, R., Milano, L. (2003). Divergence Projections for Variable Selection in Multi–layer Perceptron Networks. In: Apolloni, B., Marinaro, M., Tagliaferri, R. (eds) Neural Nets. WIRN 2003. Lecture Notes in Computer Science, vol 2859. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45216-4_32
Download citation
DOI: https://doi.org/10.1007/978-3-540-45216-4_32
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-20227-1
Online ISBN: 978-3-540-45216-4
eBook Packages: Springer Book Archive