Abstract
The Least Squares formulation of SVM (LS-SVM) finds the solution by solving a set of linear equations instead of quadratic programming implemented in SVM. The LS-SVMs provide some free parameters that have to be correctly chosen in order that the performance. Lots of tools have been developed to improve their performance, mainly the development of new classifying methods and the employment of ensembles. So, in this paper, our proposal is to use both the theory of ensembles and a genetic algorithm to enhance the LS-SVM classification. First, we randomly divide the problem into subspaces to generate diversity among the classifiers of the ensemble. So, we apply a genetic algorithm to find the values of the LS-SVM parameters and also to find the weights of the linear combination of the ensemble members, used to take the final decision.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Vapnik, V.: Statistical Learning Theory. John Wiley and Sons Inc., New York (1998)
Suykens, J.A.K., Vandewalle, J.: Least-Squares Support Vector Machine Classifiers. Neural Processing Letters 9(3) (1999)
Osuna, E., Freund, R., Girosi, F.: An Improved Training Algorithm for Support Vector Machines. In: NNSP 1997 (1997)
Lima, N., Dória Neto, A., Melo, J.: Creating an Ensemble of Diverse Support Vector Machines Using Adaboost. In: Proceedings on International Joint Conference on Neural Networks (2009)
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proceedings 13th International Conference on Machine Learning, pp. 148–156 (1996)
Ho, T.K.: The Random subspace method for constructing decision forests. IEEE Transactions Pattern Analysis and Machine Intelligence 20(8), 832–844 (1998)
Efron, B., Tibshirani, R.: An Introduction to the Bootstrap. Chapman & Hall, New York (1993)
Bryll, R., Gutierrez-Osuna, R., Quek, F.: Attribute Bagging: Improving Accuracy of Classifier Ensembles by using Random Feature Subsets. Pattern Recognition 36, 1291–1302 (2003)
Oza, N.C., Tumer, K.: Input Decimation Ensembles: Decorrelation through Dimensionality Reduction. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 238–247. Springer, Heidelberg (2001)
Ho, T.K.: The Random Subspace Method for Constructing Decision Forests. IEEE Transactions Pattern Analysis and Machine Intelligence 20, 832–844 (1998)
Ahn, H., Moon, H., Fazzari, M.J., Lim, N., Chen, J., Kodell, R.: Classification by ensembles from random partitions of high-dimensional data. Computational Statistics and Data Analysis 51, 6166–6179 (2007)
Padilha, C., Lima, N., Dória Neto, A., Melo, J.: An Genetic Approach to Support Vector Machines in classification problems. In: Proceedings on International Joint Conference on Neural Networks (2010)
Castro, L., Zuben, F.V.: Algoritmos Genéticos. Universidade Estadual de Campinas (2002), ftp://ftp.dca.fee.unicamp.br/pub/docs/vonzuben/ia707_02/topico9_02.pdf
Kuncheva, L., Whitaker, C.: Measures in diversity in classifier ensembles and their relationship with ensemble accuracy. Machine Learning 51(2), 181–207 (2003)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 993–1001 (1990)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Advances in Neural Information Processing Systems, vol. 7, pp. 231–238. MIT Press, Cambridge (1995)
Opitz, D., Shavlik, J.: Actively searching for an effective neural-network ensemble. Connection Science 8(3/4), 337–353 (1996)
Rätsch, G., Onoda, T., Müller, K.-R.: Soft Margins for Adaboost. Machine Learning 42 (2001)
Opitz, D.: Feature Selection for Ensembles. In: Proceedings of the Sixteenth National Conference on Artificial Intelligence (1999)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Padilha, C., Neto, A.D., Melo, J. (2012). Random Subspace Method and Genetic Algorithm Applied to a LS-SVM Ensemble. In: Villa, A.E.P., Duch, W., Érdi, P., Masulli, F., Palm, G. (eds) Artificial Neural Networks and Machine Learning – ICANN 2012. ICANN 2012. Lecture Notes in Computer Science, vol 7553. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33266-1_21
Download citation
DOI: https://doi.org/10.1007/978-3-642-33266-1_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33265-4
Online ISBN: 978-3-642-33266-1
eBook Packages: Computer ScienceComputer Science (R0)