Abstract
With the advancements in various data mining and social network-related approaches, datasets with a very high feature—dimensionality are often used. Various information theoretic approaches have been tried to select the most relevant set of features, and hence bring down the size of the data. Most of the times these approaches try to find a way to rank the features, so as to select or remove a fixed number of features. These principles usually assume some probability distribution for the data. These approaches also fail to capture the individual contribution of every feature in a given set of features. In this paper, we propose an approach which uses the Relief algorithm and cooperative game theory to solve the problems mentioned above. The approach was tested on NIPS 2003 and UCI datasets using different classifiers and the results were comparable to the state-of-the-art methods.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Albanese, D., Visintainer, R., Merler, S., Riccadonna, S., Jurman, G., Furlanello, C.: mlpy: Machine Learning Python (2012)
Brown, G., Pocock, A., Zhao, M., Lujan, M.: Conditional likelihood maximization: a unifying framework for information theoretic feature selection. JMLR 13, 27–66 (2012)
Cai, R., Hao, Z., Yan, X., Wen, W.: An efficient gene selection algorithm based on mututal information. Neurocomputing 72, 991–999 (2009)
Cohen, S., Dror, G., Rupin, E.: Feature selection via coalitional game theory. Neural Comput. 19, 1939–1961 (2007)
Cohen, S., Rupin, E., Dror, G.: Feature selection based on the Shapley Value. In: Proceedings of the Nineteenth IJCAI, pp. 665–670. Morgan Kaufmann Publishers (2005)
Dash, M., Liu, H.: Feature selection for classification. Intell. Data Anal. Elsevier 1(3), 131–156 (1997)
Duch, W.: Feature extraction: foundations and applications. Chapter 3, Studies in Fuzziness and Soft Computing, pp. 89–117. Springer (2006). ISBN: 3-540-35487-5
Fleuret, F.: Fast binary feature selection with conditional mutual information. JMLR 5, 1531–1555 (2004)
Frank, A., Asuncion, A.: UCI Machine Learning Repository, Irvine. University of California, School of Information and Computer Science, CA (2010)
Friedman, W.: Game Theory with Applications to Economics. Oxford University Press, New York (1990)
Gomez-Verdejo, V., Verleysen, M., Fleury, J.: Information-theoretic feature selection for functional data classification. Neurocomputing 72, 3580–3589 (2009)
Gore, S., Govindaraju, V.: Feature selection using cooperative game theory and relief algorithm. In: Skulimowski, A.M.J. (ed.) Looking into the Future of Creativity and Decision Support Systems, Krakow, Poland, 79, Nov 2013, Advances in Decision Sciences and Future Studies, vol. 2, pp. 114–125. Progress & Business Publishers, Krakow (2013)
Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L. (eds.): Feature Extraction: Foundations and Applications. Springer (2006). ISBN: 3-540-35487-5
Hild, K.E., Erdogmus, D., Torkkola, K., Principe, J.C.: Feature extraction using information theoretic learning. IEEE Trans. PAMI 28, 1385–1392 (2006)
Huang, D., Chow, T.W.S.: Effective feature selection scheme using mutual information. Neurocomputing 63, 325–343 (2005)
Jon, G.H., Kohavi, R., Peger, K.: Irrelevant features and the subset selection problem. In: Proceedings of the Eleventh ICML, pp. 121–129 (1994)
Kira, K., Rendell, L.A.: The feature selection problem: traditional methods and a new algorithm. In: Proceedings of Ninth National Conference on AI, pp. 129–134 (1992)
Kira, K., Rendell, L.A.: A practical approach to feature selection. In: Proceedings of ICML, pp. 249–256 (1992)
Kononeko, I.: Estimating attributes: ananlysis and extensions of RELIEF. ECML, pp. 101–106 (1998)
Kwak, N., Choi, C.-H.: Input feature selection by mutual information based on Parzen windows. IEEE Trans. PAMI 24, 1667–1671 (2002)
Liu, H., Yu, L.: Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. KDE 17(4), 491–501 (2005)
Liu, J., Lee, S.: Study on feature select based on coalition game. In: Proceedings of the International Conference on Neural Networks and Signal Processing, pp. 445–450 (2008)
Narendra, P. M., Fukunaga, K.: A branch and bound algorithm for feature selection. IEEE Trans. Comput. C-26(9), 917–922 (1977)
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance and min-redundancy. IEEE Trans. PAMI 27, 1226–1238 (2005)
Robnik Sikonja, M., and Kononenko, I.: An adaptation of relief for attribute estimation in regression. In: Proceedings of the Fourteenth ICML, pp. 296–304 (1997)
Shapley, L.S.: A value for n-person games. In: Kuhn, A.W.T.H.W. (ed.) In Contributions to the Theory of Games, vol. 2. Princeton University Press, New York (1990)
Sun, X., Liu, Y., Li, J., Zhu, J., Liu, X., Chen, H.: Using cooperative game theory to optimize the feature selection problem. Neurocomputing 97, 86–93 (2012)
Sun, X., Liu, Y., Li, J., Zhu, J., Liu, X., Chen, H.: Feature evaluation and selection with cooperative game theory. Pattern Recogn. 45, 2992–3002 (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Gore, S., Govindaraju, V. (2016). Feature Selection Using Cooperative Game Theory and Relief Algorithm. In: Skulimowski, A., Kacprzyk, J. (eds) Knowledge, Information and Creativity Support Systems: Recent Trends, Advances and Solutions. Advances in Intelligent Systems and Computing, vol 364. Springer, Cham. https://doi.org/10.1007/978-3-319-19090-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-19090-7_30
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19089-1
Online ISBN: 978-3-319-19090-7
eBook Packages: Computer ScienceComputer Science (R0)