Abstract
The reduction of input dimensionality is an important subject in modelling, knowledge discovery and data mining. Indeed, an appropriate combination of inputs is desirable in order to obtain better generalisation capabilities with the models. There are several approaches to perform input selection. In this work we will deal with techniques guided by measures of input relevance or input sensitivity. Six strategies to assess input relevance were tested over four benchmark datasets using a backward selection wrapper. The results show that a group of techniques produces input combinations with better generalisation capabilities even if the implemented wrapper does not compute any measure of generalisation performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bellman, R.: Adaptive control processes. A guided tour, Princeton University Press, New Jersey (1961)
Bishop, C.: Neural networks for pattern recognition. Oxford University Press, New York (1995)
Box, G., Jenkins, G.: Time Series Analysis, Forecasting and Control. Holden Day, San Francisco (1970)
Chambers, J.M., Cleveland, W.S., Kleiner, B., Tukey, P.A.: Graphical Methods for Data Analysis. Wadsworth & Brooks/Cole, 62 (1983)
Chapados, N., Bengio, Y.: Input decay: simple and effective soft variable selection. Neural Networks. In: Proc. IJCNN’01. Int. Joint Conference, vol. 2, pp. 1233–1237 (2001)
Cibas, T., Fogelman, F., Gallinari, P., Raudys, S.: Variable selection with optimal cell damage. In: ICANN’94. Int. Conf. on Artificial Neural Networks, Amsterdam (1994)
Forina, M.: PARVUS - An Extendible Package for Data Exploration, Classification and Correlation. Institute of Pharmaceutical and Food Analysis and Technologies, Via Brigata Salerno, 16147 Genoa, Italy (1991)
Garson, G.D.: Interpreting neural network connection weights. Artificial Intelligence Expert 6, 47–51 (1991)
Gevrey, M., Dimopoulos, I., Lek, S.: Review and comparison of methods to study the contribution of variables in artificial neural network models. Ecological Modelling 160, 249–264 (2003)
Gorman, R., Sejnowski, T.: Analysis of Hidden Units in a Layered Network Trained to Classify Sonar Targets. Neural Networks 1, 75–89 (1988)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. The Journal of Machine Learning Research 3, 1157–1182 (2003)
Haykin, S., Neural Networks, A.: Comprehensive Foundation. Macmillan College Publishing, New York (1994)
Jang, J.-S.R.: Input selection for ANFIS learning. In: Proceedings of the Fifth IEEE International Conference on Fuzzy Systems, vol. 2, pp. 1493–1499. IEEE Computer Society Press, Los Alamitos (1996)
Kaur, K., Kaur, A., Malhotra, R.: Alternative Methods to Rank the Impact of Object Oriented Metrics in Fault Prediction Modeling using Neural Networks. Transactions on Engineering, Computing and Technology 13, 207–212 (2006)
Kohavi, R., John, G.: Wrappers for feature selection. Artificial Intelligence, 232–273 (1997)
Merz, C., Murphy, P.: UCI Repository of machine learning databases. Irvine, CA: University of California, Department of Information and Computer Science (1998), http://www.ics.uci.edu/~mlearn/MLRepository.html
Olden, J., Jackson, D.: Illuminating the “black box”: a randomization approach for understanding variable contributions in artificial neural networks. Ecological Modelling 154, 135–150 (2002)
Olden, J., Joy, M., Death, R.: An accurate comparison of methods for quantifying variable importance in artificial neural networks using simulated data. Ecological Modelling 178, 389–397 (2004)
Schuschel, D., Hsu, C.: A Weight Analysis Based Wrapper Approach to Neural Nets Feature Subset Selection. IEEE Transactions on Systems, Man and Cybernetics 32, 207–221 (2002)
Setiono, R., Liu, H.: Neural-network feature selector. IEEE transactions on neural networks 8, 654–662 (1997)
Stracuzzi, D., Utgoff, P.: Randomized Variable Elimination. The Journal of Machine Learning Research 5, 1331–1362 (2004)
Sugeno, M., Yasukawa, T.: A Fuzzy-Logic-Based Approach to Qualitative Modelling. IEEE Transactions on Fuzzy Systems 1, 7–31 (1993)
Witten, I.H., Frank, E.: Data Mining. Practical Machine Learning Tools and Techniques, 2nd edn. Elsevier, Morgan Kaufmann publishers (2005)
Zurada, J., Malinowski, A., Usui, S.: Perturbation method for deleting redundant inputs of perceptron networks. Neurocomputing 14, 177–193 (1997)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Satizábal M., H.F., Pérez-Uribe, A. (2007). Relevance Metrics to Reduce Input Dimensions in Artificial Neural Networks. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4_5
Download citation
DOI: https://doi.org/10.1007/978-3-540-74690-4_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74689-8
Online ISBN: 978-3-540-74690-4
eBook Packages: Computer ScienceComputer Science (R0)