Abstract
In the framework of fuzzy rule-based models for regression problems, we propose a novel approach to feature selection based on the minimal-redundancy-maximal-relevance criterion. The relevance of a feature is measured in terms of a novel definition of fuzzy mutual information between the feature and the output variable. The redundancy is computed as the average fuzzy mutual information between the feature and the just selected features. The approach results to be particularly suitable for selecting features before designing fuzzy rule-based systems (FRBSs). We tested our approach on twelve regression problems using Mamdani FRBSs built by applying the Wang and Mendel algorithm. We show that our approach is particularly effective in selecting features by comparing the mean square errors achieved by the Mamdani FRBSs generated using the features selected by a state of the art feature selection algorithm and by our approach.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Alcala, R., Ducange, P., Herrera, F., Lazzerini, B., Marcelloni, F.: A multiobjective evolutionary approach to concurrently learn rule and data bases of linguistic fuzzy-rule-based systems. IEEE Transactions on Fuzzy Systems 17(5), 1106–1122 (2009)
Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks 5, 537–550 (1994)
Dash, M., Liu, H.: Feature selection for classification. Intelligent Data Analysis 1, 131–156 (1997)
Estévez, P., Tesmer, M., Perez, C., Zurada, J.: Normalized mutual information feature selection. IEEE Transaction on Neural Networks 20(2), 189–201 (2009)
Gerontidis, I., Petasakis, I.E.: Lumpability of absorbing markov chains and replacement chains on fuzzy partitions. In: FUZZ-IEEE, pp. 1–8 (2010)
Hall, M.A., Smith, L.A.: Practical Feature Subset Selection for Machine Learning (1998)
Huawen, L., Jigui, S., Lei, L., Huijie, Z.: Feature selection with dynamic mutual information. Pattern Recogn. 42(7), 1330–1339 (2009)
Kojadinovic, I.: Relevance measures for subset variable selection in regression problems based on k-additive mutual information. Comput. Stat. Data Anal. 49(4), 1205–1227 (2005)
Kudo, M., Sklansky, J.: Comparison of algorithms that select features for pattern classifiers. Pattern Recognition 33(1), 25–41 (2000)
Kuriyama, K.: Entropy of a finite partition of fuzzy sets. Journal of Mathematical Analysis and Applications 94, 38–43 (1983)
Rossi, F., Lendasse, A., Franc̨ois, D., Wertz, V., Verleysen, M.: Mutual information for the selection of relevant variables in spectrometric nonlinear modelling. Chemometrics and Intelligent Laboratory Systems 80(2), 215–226 (2006)
Sánchez, L., Suárez, M.R., Villar, J.R., Couso, I.: Mutual information-based feature selection and partition design in fuzzy rule-based classifiers from vague data. Int. J. Approx. Reasoning 49(3), 607–622 (2008)
Sheskin, D.J.: Handbook of Parametric and Nonparametric Statistical Procedures, 4th edn. Chapman & Hall/CRC (2007)
Wang, L., Mendel, J.: Generating fuzzy rules by learning from examples. IEEE Transactions on Systems Man and Cybernetics 22(6), 1414–1427 (1992)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this paper
Cite this paper
Antonelli, M., Ducange, P., Marcelloni, F. (2013). Feature Selection Based on Fuzzy Mutual Information. In: Masulli, F., Pasi, G., Yager, R. (eds) Fuzzy Logic and Applications. WILF 2013. Lecture Notes in Computer Science(), vol 8256. Springer, Cham. https://doi.org/10.1007/978-3-319-03200-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-319-03200-9_4
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-03199-6
Online ISBN: 978-3-319-03200-9
eBook Packages: Computer ScienceComputer Science (R0)