Abstract
Feature selection is frequently used in data pre-processing for data mining. When the training data set is too large, sampling is commonly used to overcome the difficulty. This work investigates the applicability of active sampling in feature selection in a filter model setting. Our objective is to partition data by taking advantage of class information so as to achieve the same or better performance for feature selection with fewer but more relevant instances than random sampling. Two versions of active feature selection that employ class information are proposed and empirically evaluated. In comparison with random sampling, we conduct extensive experiments with benchmark data sets, and analyze reasons why class-based active feature selection works in the way it does. The results will help us deal with large data sets and provide ideas to scale up other feature selection algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
R. Kohavi and G.H. John. Wrappers for feature subset selection. Artificial Intelligence, 97(1–2):273–324, 1997.
H. Liu and H. Motoda. Feature Selection for Knowledge Discovery & Data Mining. Boston: Kluwer Academic Publishers, 1998.
M. Dash and H. Liu. Feature selection methods for classifications. Intelligent Data Analysis: An International Journal, 1(3), 1997.
L. Talavera. Feature selection as a preprocessing step for hierarchical clustering. In Proceedings of Internationl Conference on Machine Learning (ICML’99), 1999.
U.M. Fayyad and K.B. Irani. The attribute selection Problem in decision tree generation. In AAAI-92, Proceedings of the Ninth National Conference on Artificial Intelligence, pages 104–110. AAAI Press/The MIT Press, 1992.
G.H. John, R. Kohavi, and K. Pfleger. Irrelevant feature and the subset selection Problem. In W.W. Cohen and Hirsh H., editors, Machine Learning: Proceedings of the Eleventh International Conference, pages 121–129, New Brunswick, N.J., 1994. Rutgers University.
P. Langley. Selection of relevant features in machine learning. In Proceedings of the AAAI Fall Symposium on Relevance. AAAI Press, 1994.
P.S. Bradley and O. L. Mangasarian. Feature selection via concave minimization and support vector machines. In Proceedings of Fifteenth International Conference on Machine Learning, pages 82–90, 1998.
M.A. Hall. Correlation-based feature selection for discrete and numeric class machine learning. In Proceedings of Seventeenth International Conference on Machine Learning (ICML-00). Morgan Kaufmann Publishers, 2000.
W.G. Cochran. Sampling Techniques. John Wiley & Sons, 1977.
B. Gu, F. Hu, and H. Liu. Sampling: Knowing Whole from Its Part, pages 21–38. Boston: Kluwer Academic Publishers, 2001.
K. Kira and L.A. Rendell. The feature selection Problem: Traditional methods and a new algorithm. In Proceedings of the Tenth National Conference on Artificial Intelligence, pages 129–134. Menlo Park: AAAI Press/The MIT Press, 1992.
I. Kononenko. Estimating attributes: Analysis and extension of RELIEF. In F. Bergadano and L. De Raedt, editors, Proceedings of the European Conference on Machine Learning, April 6–8, pages 171–182, Catania, Italy, 1994. Berlin: Springer.
J.R. Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann, 1993.
I.H. Witten and E. Frank. Data Mining — Practical Machine Learning Tools and Techniques with JAVA Implementations. Morgan Kaufmann Publishers, 2000.
C.L. Blake and C.J. Merz. UCI repository of machine learning databases, 1998. http://www.ics.uci.edu/~mlearn/MLRepository.html.
S.D. Bay. The UCI KDD archive, 1999. http://kdd.ics.uci.edu.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Liu, H., Yu, L., Dash, M., Motoda, H. (2003). Active Feature Selection Using Classes. In: Whang, KY., Jeon, J., Shim, K., Srivastava, J. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2003. Lecture Notes in Computer Science(), vol 2637. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-36175-8_48
Download citation
DOI: https://doi.org/10.1007/3-540-36175-8_48
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-04760-5
Online ISBN: 978-3-540-36175-6
eBook Packages: Springer Book Archive