Abstract
In applications of learning from examples to real-world tasks, feature subset selection is important to speed up training and to improve generalization performance. Ideally, an inductive algorithm should use subset of features as small as possible. In this paper however, the authors show that the problem of selecting the minimum subset of features is NP-hard. The paper then presents a greedy algorithm for feature subset selection. The result of running the greedy algorithm on hand-written numeral recognition problem is also given.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Almuallim H, Dietterich T G. Learning with many irrelevant features. InAAAI-91 Proc., 9th National Conference on Artificial Intelligence, 1991.
Kira K, Rendell L. The feature selection problem: Traditional methods and a new algorithm. InAAAI-92 Proc., 10th International Conference on Artificial Intelligence, 1992.
Caruana R, Freitag D. Greedy Attribute Selection. ML’94, Rutgers University, New Brunswick, NJ, pp.28–36, 1994.
Littlestone N. Learning quickly when irrelevant attributes abound.Machine Learing, 2(4): 285–318.
John G, Kohavi R, Pfleger K. Irrelevant features and the subset selection problem. InProc. of the 11th International Conference on Machine Learning, 1994.
Draper N, Smith H. Applied Regression Analysis. John Wiley & Sons, 2nd edition, 1981.
Breiman L, Friedman J, Olshen R, Stone C. Classification and Regression Trees. Wadsworth International Group, 1984.
Li X Y, Hong J, R. Error-correcting form class matching for reading system. InSPIEEE Conference on Visual Communication and Image Processing. Vol2308, 1994.
Hong J R. AE1: An extension matrix approximate method for the general covering problem.Int. Journal of Computer and Information Science, 1985, 14(6): 421–437.
Wu X. HCV: A heuristic covering algorithm for extension matrix approach. DAI Research Paper No.578, Dept. of Artificial Intelligence, Univ. of Edinburgh. 1992.
D’Andrea V, Uhrik C. Cellular automata implementation of an extension matrix algorithm. InProc. of Summer School on Machine Learning, Sogesta, Urbino, Italy, 1989.
Johnson D S. Approximation algorithms for combinational problem.J. of Computer and Systems Science, 1973, 9(3): 38–48.
Michalski R S, Mozetic I, Hong J R, Lavrac N. The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. InProc. of AAAI 86 Philadelphia, Pennsylvania, 1986.
Quinlan J R. Induction of decision trees.Machine Learning, 1986, 1: 81–106.
Author information
Authors and Affiliations
Additional information
Chen Bin, born in 1968, received his M.S. degree in 1994. He is now a Ph.D. candidate in the Department of Computer Science, Harbin Institute of Technology. His research focuses on machine learning and computational learning theory.
Hong Jiarong, born in 1939, is a Professor of the Department of Computer Science, Harbin Institute of Technology. His research focuses on machine learning, computational learning theory, expert system and pattern recognition.
Wang Yiadong, born in 1964, is now a Ph.D. candidate in the Department of Management Engineering. His research focuses on expert system and case-based reasoning
Rights and permissions
About this article
Cite this article
Chen, B., Hong, J. & Wang, Y. The minimum feature subset selection problem. J. of Comput. Sci. & Technol. 12, 145–153 (1997). https://doi.org/10.1007/BF02951333
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02951333