Abstract
XCSR is an accuracy-based learning classifier system which can handle classification problems with real-value features. However, as the number of features increases, a high classification accuracy comes at the cost of more resources: larger population sizes and longer computational running times. In this paper we investigate PCA-XCSR (a sequential application of PCA and XCSR) in three environments with different characteristics: a discrete and imbalanced environment (KDD’99 network intrusion), a continuous and highly symmetric environment (MiniBooNE), and a highly discrete, highly imbalanced environment (Census/Income (KDD)). These experiments show that in the three different environments, PCA-XCSR, in addition to being able to reduce the computational resources and time requirements of XCSR by approximately 50 %, is able to consistently maintain its high accuracy. In addition to that, it reduces the required population size needed by XCSR. Also, we suggest heuristics for selecting the number of principal components to use when using PCA-XCSR.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Abdi H, Williams LJ (2010) Principal component analysis. Wiley Interdis Rev Comput Stat 2(4):433–459
Abedini M, Kirley M (2009) CoXCS: a coevolutionary learning classifier based on feature space partitioning. In: Nicholson A, Li X (eds) AI 2009: advances in artificial intelligence, lecture notes in computer science, vol 5866. Springer, Berlin, pp 360–369
Abedini M, Kirley M (2010) A multiple population XCS: evolving condition-action rules based on feature space partitions. In: Evolutionary computation (CEC), 2010 IEEE congress, pp 1–8
Bacardit J, Stout M, Hirst J, Krasnogor N (2008) Data mining in proteomics with learning classifier systems. In Bull L, Bernad-Mansilla E, Holmes J (eds) Learning classifier systems in data mining, studies in computational intelligence, vol 125. Springer, Berlin, pp 17–46
Behdad M, Barone L, French T, Bennamoun M (2010) An investigation of real-valued accuracy-based learning classifier systems for electronic fraud detection. In: Proceedings of the 12th annual conference companion on Genetic and evolutionary computation, pp 1893–1900
Bouzida Y, Cuppens F, Cuppens-Boulahia N, Gombault S (2004) Efficient intrusion detection using principal component analysis. In: Proceedings of the 3ème Conférence sur la Sécurité et Architectures Réseaux (SAR)
Bull L, Bernadó-Mansilla E, Holmes J (2008) Learning classifier systems in data mining. Studies in computational intelligence. Springer, Berlin
Butz MV, Wilson SW (2000) An algorithmic descrtipion of XCS. Technical report 2000017, Illinois Genetics Algorithms Laboratory
Depren O, Topallar M, Anarim E, Ciliz MK (2005) An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks. Exp Syst Appl 29(4):713–722
Golovko V, Vaitsekhovich L, Kochurko P, Rubanau U (2007) Dimensionality reduction and attack recognition using neural network approaches. In: Neural networks, 2007. IJCNN 2007. International joint conference, pp 2734–2739
Hettich S, Bay SD (1999) KDD’99 network intrusion detection data set. UCI Machine Learning Repository [http://archive.ics.uci.edu/ml/]
Holland JH (1976) Adaptation. Prog Theor Biol 4:263–293
Kirley M, Abedini M (2009) CoXCS: a CoEvolutionary learning classifier based on feature space partitioning. In: Lecture notes in computer science, vol 5866, pp 360–369
Lane T, Kohavi R (2010) Census-Income (KDD) data set. UCI machine learning repository [http://archive.ics.uci.edu/ml/]
Moore B (1981) Principal component analysis in linear systems: controllability, observability, and model reduction. IEEE Trans Autom Control 26:17–32
O’Toole AJ, Abdi H, Deffenbacher KA, Valentin D (1993) Low-dimensional representation of faces in higher dimensions of the face space. J Opt Soc Am 10(3):405–411
Roe B (2010) MiniBooNE particle identification data set. UCI machine learning repository [http://archive.ics.uci.edu/ml/]
Shyu M, Chen S, Sarinnapakorn K, Chang L (2003) A novel anomaly detection scheme based on principal component classifier. In: Proceedings of the IEEE foundations and new directions of data mining workshop, pp 172–179
Sigaud O, Wilson SW (2007) Learning classifier systems: a survey. Soft computing—a fusion of foundations. Method Appl 11(11):1065–1078
Stone C, Bull L (2003) For real! XCS with continuous-valued inputs. Evol Comput 11(3):299–336
Tsai WC, Chen AP (2011) Using the XCS classifier system for portfolio allocation of MSCI index component stocks. Exp Syst Appl 38(1):151–154
Urbanowicz RJ, Moore JH (2009) Learning classifier systems: a complete introduction, review, and roadmap. J Artif Evol App 2009:1:1–1:25
Wada A, Takadama K, Shimohara K, Katai O (2007) Analyzing parameter sensitivity and classifier representations for real-valued XCS. In: Learning classifier systems, lecture notes in computer science, vol 4399. Springer, Berlin, pp 1–16
Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175
Wilson SW (2000) Get real! XCS with continuous-valued inputs. In: Learning classifier systems, from foundations to applications. Springer, Berlin, pp 209–222
Acknowledgments
The first author would like to acknowledge the financial support provided by the Robert and Maude Gledden Scholarship.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Behdad, M., French, T., Barone, L. et al. On principal component analysis for high-dimensional XCSR. Evol. Intel. 5, 129–138 (2012). https://doi.org/10.1007/s12065-012-0075-6
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12065-012-0075-6