Abstract
When using a Genetic Algorithm (GA) to optimize the feature space of pattern classification problems, the performance improvement is not only determined by the data set used, but also depends on the classifier. This work compares the improvements achieved by GA-optimized feature transformations on several simple classifiers. Some traditional feature transformation techniques, such as Principal Components Analysis (PCA) and Linear Discriminant Analysis (LDA) are also tested to see their effects on the GA optimization. The results based on some real-world data and five benchmark data sets from the UCI repository show that the improvements after GA-optimized feature transformation are in reverse ratio with the original classification rate if the classifier is used alone. It is also shown that performing the PCA and LDA transformations on the feature space prior to the GA optimization improved the final result.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
M. Pei, E. D. Goodman, W. F. Punch. “Feature Extraction Using Genetic Algorithms”, Proceeding of International Symposium on Intelligent Data Engineering and Learning’98 (IDEAL’98) Hong Kong, Oct. 1998.
David B. Skalak. “Using a Genetic Algorithm to Learn Prototypes for Case Retrieval and Classification”, Proceedings of the AAAI-93 Case-Based Reasoning Workshop, pages 64–69, Washington, D.C., American Association for Artificial Intelligence, Menlo Park, CA, 1994.
W.F. Punch, E.D. Goodman, Min Pei, Lai Chia-Shun, P. Hovland and R. Enbody. “Further Research on Feature Selection and Classification Using Genetic Algorithms”, In 5th International Conference on Genetic Algorithms, Champaign IL, pages 557–564, 1993
Hiram A. Firpi Cruz. “Genetically Found, Neurally Computed Artificial Features With Applications to Epileptic Seizure Detection and Prediction”, Master’s Thesis, University of Puerto Rico, Mayaguez, 2001.
M. Prakash, M. Narasimha Murty. “A Genetic Approach for Selection of (Near-) Optimal Subsets of Principal Components for Discrimination”, Pattern Recognition Letters, Vol. 16, pages 781–787, 1995.
Blake, C.L. & Merz, C.J. UCI Repository of machine learning databases [http://www.ics.uci.edu/~mlearn/MLRepository.html]. Irvine, CA: University of California, Department of Information and Computer Science (1998).
S. Bandyopadhyay, C.A. Murthy. “Pattern Classification Using Genetic Algorithms”, Pattern Recognition Letters, Vol. 16, pages 801–808, 1995.
R. Srikanth, R. George, N. Warsi, D. Prabhu, F.E. Petry, B.P. Buckles. “A Variable-Length Genetic Algorithm for Clustering and Classification”, Pattern Recognition Letters, Vol. 16, pages 789–800, 1995.
W. Siedlecki, J. Sklansky. “A Note on Genetic Algorithms for Large-Scale Feature Selection”, Pattern Recognition Letters, Vol. 10, pages 335–347, 1989.
Ludmila I. Kuncheva. “Editing for the k-Nearest Neighbors Rule by a Genetic Algorithm”, Pattern Recognition Letters, Vol. 16, pages 809–814, 1995.
Richard O. Duda, Peter E. Hart, David G. Stock. “Pattern Classification”, Second Edition, Wiley 2001.
Ludmila I. Kuncheva and Lakhmi C. Jain. “Designing Classifier Fusion Systems by Genetic Algorithms”, IEEE Transactions on Evolutionary Computation, Vol. 4, No. 4, September 2000.
Ron Kohavi, George John. “The Wrapper Approach”, Feature Extraction, Construction. and Selection: A Data Mining Perspective. Edited by Hiroshi Motoda, Huan Liu, Kluwer Academic Publishers, July 1998.
George H. John, Ron Kohavi, Karl Pfleger. “Irrelevant Features and the Subset Selection Problem”, Proceedings of the Eleventh International Conference of Machine Learning, pages 121–129, Morgan Kaufmann Publishers, San Francisco, CA, 1994.
Zhijian Huang. “Genetic Algorithm Optimized Feature Extraction and Selection for ECG Pattern Classification”, Master’s Thesis, Michigan State University, 2002.
O.L. Mangasarian, W.N. Street and W.H. Wolberg. “Breast Cancer Diagnosis and Prognosis via Linear Programming”, Operations Research, 43(4), pages 570–577, July–August 1995.
Gorman and T. J. Sejnowski. “Learned Classification of Sonar Targets Using Massively Parallel Network”, IEEE Transactions on Acoustic, Speech and Signal Processing, 36(7), pages 1135–1140, 1988.
V.G. Sigilito, S.P. Wing, L.V. Hutton and K.B. Baker. “Classification of Radar Returns from the Ionosphere Using Neural Networks”, Johns Hopkins APL Technical Digest, Vol. 10, pages 262–266, 1989.
Thomas Gartner, Peter A. Flach. “WBCSVM: Weighted Bayesian Classification based on Support Vector Machine”, Eighteenth International Conference on Machine Learning (ICML-2001), pages 156–161. Morgan Kaufmann, 2001.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Huang, Z., Pei, M., Goodman, E., Huang, Y., Li, G. (2003). Genetic Algorithm Optimized Feature Transformation — A Comparison with Different Classifiers. In: Cantú-Paz, E., et al. Genetic and Evolutionary Computation — GECCO 2003. GECCO 2003. Lecture Notes in Computer Science, vol 2724. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45110-2_108
Download citation
DOI: https://doi.org/10.1007/3-540-45110-2_108
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40603-7
Online ISBN: 978-3-540-45110-5
eBook Packages: Springer Book Archive