Abstract
When used for data reduction, feature selection may successfully identify and discard irrelevant attributes, and yet fail to improve learning accuracy because regularities in the concept are still opaque to the learner. In that case, it is necessary to highlight regularities by constructing new characteristics that abstract the relations among attributes. This paper highlights the importance of feature construction when attribute interaction is the main source of learning difficulty and the underlying target concept is hard to discover by a learner using only primitive attributes. An empirical study centered on predictive accuracy shows that feature construction significantly outperforms feature selection because, even when done perfectly, detection of interacting attributes does not sufficiently facilitates discovering the target concept.
Supported by the Spanish Ministry of Science and Technology, TIN2008-02081.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Liu, H., Motoda, H. (eds.): Feature Extraction, Construction and Selection: A Data Mining Perspective, vol. 453. Kluwer Academic Publishers, MA (1998)
Liu, H., Motoda, H. (eds.): Computational Methods of Feature Selection. Data Mining and Knowledge Discovery Series. Chapman & Hall/CRC, Boca Raton (2007)
Shafti, L.S., Pérez, E.: Fitness function comparison for GA-based feature construction. In: Borrajo, D., Castillo, L., Corchado, J.M. (eds.) CAEPIA 2007. LNCS, vol. 4788, pp. 249–258. Springer, Heidelberg (2007)
Shafti, L.S., Pérez, E.: Data reduction by genetic algorithms and non-algebraic feature construction: A case study. In: Proceedings of HIS (2008)
Blake, C., Merz, C.: UCI repository of machine learning databases (1998)
Holte, R.C.: Very simple classification rules perform well on most commonly used datasets. Machine Learning 11, 63–91 (1993)
Jakulin, A., Bratko, I., et al.: Attribute interactions in medical data analysis. In: Dojat, M., Keravnou, E.T., Barahona, P. (eds.) AIME 2003. LNCS, vol. 2780, pp. 229–238. Springer, Heidelberg (2003)
Danyluk, A.P., Provost, F.J.: Small disjuncts in action: Learning to diagnose errors in the local loop of the telephone network. In: Proceedings of ICML (1993)
Rendell, L.A., Seshu, R.: Learning hard concepts through constructive induction: Framework and rationale. Computational Intelligence 6, 247–270 (1990)
Freitas, A.A.: Understanding the crucial role of attribute interaction in data mining. Artificial Intelligence Review 16(3), 177–199 (2001)
Zhao, Z., Liu, H.: Searching for interacting features. In: Veloso, M.M. (ed.) Proceedings of IJCAI, Hyderabad, India, pp. 1156–1161 (January 2007)
Bloedorn, E., Michalski, R.S.: Data-driven constructive induction: Methodology and applications. In: [1], pp. 51–68
Michalski, R.S.: Pattern recognition as knowledge-guided computer induction. Technical Report 927, Dept. of Computer Science, University of Illinois (1978)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Vafaie, H., DeJong, K.: Feature space transformation using genetic algorithms. IEEE Intelligent Systems 13(2), 57–65 (1998)
Pérez, E., Rendell, L.A.: Using multidimensional projection to find relations. In: Proceedings of the Twelfth ICML, pp. 447–455. Morgan Kaufmann, San Francisco (1995)
Pazzani, M.: Constructive induction of cartesian product attributes. In: [1], pp. 341–354
Zupan, B., Bratko, I., et al.: Function decomposition in machine learning. Machine Learning and Its Applications, Advanced Lectures, 71–101 (2001)
Grunwald, P.D.: The Minimum Description Length Principle. MIT Press, Cambridge (2007)
Shafti, L.S.: Multi-feature construction based on genetic algorithms and non-algebraic feature representation to facilitate learning concepts with complex interactions. Ph.D thesis, EPS, Universidad Autonoma de Madrid (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Shafti, L.S., Pérez, E. (2009). Feature Construction and Feature Selection in Presence of Attribute Interactions. In: Corchado, E., Wu, X., Oja, E., Herrero, Á., Baruque, B. (eds) Hybrid Artificial Intelligence Systems. HAIS 2009. Lecture Notes in Computer Science(), vol 5572. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-02319-4_71
Download citation
DOI: https://doi.org/10.1007/978-3-642-02319-4_71
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-02318-7
Online ISBN: 978-3-642-02319-4
eBook Packages: Computer ScienceComputer Science (R0)