Abstract
In feature selection (FS), different strategies usually lead to different results. Even the same strategy may do so in distinct feature selection contexts. We propose a feature subspace ensemble method, consisting on the parallel combination of decisions from multiple classifiers. Each classifier is designed using variations of the feature representation space, obtained by means of FS. With the proposed approach, relevant discriminative information contained in features neglected in a single run of a FS method, may be recovered by the application of multiple FS runs or algorithms, and contribute to the decision through the classifier combination process. Experimental results on benchmark data show that the proposed feature subspace ensembles method consistently leads to improved classification performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Heijden, V., et al.: Classification, parameter estimation and state estimation - an engineering approach using MATLAB. John Wiley & Sons, Chichester (2004)
Duda, R., Hart, P., Stork, D.: Pattern classification, 2nd edn. John Wiley & Sons, Chichester (2001)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. Journal of Machine Learning Research 3, 1157–1182 (2003), http://www.jmlr.org/papers/v3/guyon03a.html
Kudo, M., Sklansky, J.: Comparison of algorithms that select features for pattern classifiers. Pattern Recognition 33, 25–41 (2000), citeseer.ist.psu.edu/kudo00comparison.html
Jain, A., Zongker, D.: Feature selection: Evaluation, application, and small sample performance. IEEE Transactions on Pattern Analysis and Machine Intelligence 19(2), 153–158 (1997), http://ilk.uvt.nl/~tbogers/research/jain-1997.pdf
Kohavi, R., John, G.: Wrappers for feature subset selection. Artificial Intelligence 97(1-2), 273–324 (1997), http://citeseer.ist.psu.edu/kohavi96wrappers.html
Skurichina, M., Duin, R.: Combining feature subsets in feature selection. In: Oza, N.C., et al. (eds.) MCS 2005. LNCS, vol. 3541, pp. 165–175. Springer, Heidelberg (2005)
Reunanen, J.: Overfitting in making comparisons between variable selection methods. Journal of Machine Learning Research 3, 1371–1382 (2003), http://jmlr.csail.mit.edu/papers/volume3/reunanen03a/reunanen03a.pdf
Kittler, J., et al.: On combining classifiers. IEEE Transactions on Pattern Analysis and Machine Intelligence 20, 226–239 (1998)
Duin, R., Tax, D.: Experiments with classifier combining rules. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, Springer, Heidelberg (2000)
Lam, L.: Classifier combinations: Implementation and theoretical issues. In: Kittler, J., Roli, F. (eds.) MCS 2000. LNCS, vol. 1857, pp. 78–86. Springer, Heidelberg (2000)
Grimaldi, M., Cunningham, P., Kokaram, A.: An evaluation of alternative feature selection strategies and ensemble techniques for classifying music (May 2003)
Russell, S., Norvig, P.: Artificial Intelligence: A Modern Approach, 2nd edn. Prentice-Hall, Englewood Cliffs (2002)
Silva, H.: Feature selection in pattern recognition systems. Master’s thesis, Universidade Técnica de Lisboa, Instituto Superior Técnico (2007)
Tax, D., Duin, R.: Using two-class classifiers for multiclass classification. In: International Conference on Pattern Recognition, Quebec, Canada (2002)
Fred, A.: Finding consistent clusters in data partitions. In: Kittler, J., Roli, F. (eds.) MCS 2001. LNCS, vol. 2096, pp. 309–318. Springer, Heidelberg (2001)
Lam, L., Suen, S.: Application of majority voting to pattern recognition: An analysis of its behavior and performance. IEEE Transactions on Systems, Man, and Cybernetics 27, 553–568 (1997), http://ieeexplore.ieee.org/iel1/3468/13434/00618255.pdf
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Silva, H., Fred, A. (2007). Feature Subspace Ensembles: A Parallel Classifier Combination Scheme Using Feature Selection. In: Haindl, M., Kittler, J., Roli, F. (eds) Multiple Classifier Systems. MCS 2007. Lecture Notes in Computer Science, vol 4472. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72523-7_27
Download citation
DOI: https://doi.org/10.1007/978-3-540-72523-7_27
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72481-0
Online ISBN: 978-3-540-72523-7
eBook Packages: Computer ScienceComputer Science (R0)