Abstract
We propose in this chapter a new method to score subsets of variables according to their usefulness for a given model. It can be qualified as a variable ranking method ‘in the context of other variables’. The method consists in replacing a variable value by another value obtained by randomly choosing a among other values of that variable in the training set. The impact of this change on the output is measured and averaged over all training examples and changes of that variable for a given training example. As a search strategy, backward elimination is used. This method is applicable on every kind of model and on classification or regression task. We assess the efficiency of the method with our results on the NIPS 2003 feature selection challenge.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
W. G. Baxt and H. White. Bootstrapping confidence intervals for clinical inputs variable effects in a network trained to identify the presence of acute myocardial infraction. Neural Computation, 7:624–638, 1995.
Leo Breiman. Random forest. Machine Learning, 45, 2001.
Anthony N. Burkitt. Refined pruning techniques for feed-forward neural networks. Complex Systems, 6:479–494, 1992.
Raphael Féraud and Fabrice Clérot. A methodology to explain neural network classification. Neural Networks, 15:237–246, 2002.
Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection. JMLR, 3 (Mar):1157–1182, 2003.
R. Kohavi and G. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1–2), 1997.
Peter S. Maybeck. Stochastic models, estimation, and control, volume 141 of Mathematics in Science and Engineering. 1979.
J. Moody. Prediction Risk and Architecture Selection for Neural Networks. From Statistics to Neural Networks-Theory and Pattern Recognition. Springer-Verlag, 1994.
A. N. Réfénes, A. Zapranis, and J. Utans. Stock performance using neural networks: A comparative study with regression models. Neural Network, 7:375–388, 1994.
D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning internal representations by error propagation. In Parallel Distribued Processing: Explorations in the Microstructures of Cognition, volume 1, 1986.
Greg Welch and Gary Bishop. An introduction to the kalman filter. In SIGGRAPH, Los Angeles, August 12–17 2001. Course 8.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Lemaire, V., Clérot, F. (2006). An Input Variable Importance Definition based on Empirical Data Probability Distribution. In: Guyon, I., Nikravesh, M., Gunn, S., Zadeh, L.A. (eds) Feature Extraction. Studies in Fuzziness and Soft Computing, vol 207. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-35488-8_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-35488-8_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-35487-1
Online ISBN: 978-3-540-35488-8
eBook Packages: EngineeringEngineering (R0)