Abstract
Variable and feature selection has been a research topic with practical significance in many areas such as statistics, pattern recognition, machine learning and data mining. The task of feature selection is to choose an effective feature subset out of a given feature set to reduce the feature space dimensionality. In this paper, along with the guidelines of Energy-based model, a unified energy-based framework for feature selection and a feature ranking algorithm under this framework is presented. On the other hand, in order to increase the stability of our algorithm, an ensemble feature selection is introduced. Some experiments are conducted on the real world and synthesis data sets to demonstrate the ability of our feature selection algorithm and the stability improvement of the ensemble feature selection.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Liu, H., Yu, L.: Toward Integrating Feature Selection Algorithms for Classification and Clustering. IEEE Trans. Knowledge and Data Engineering 17, 494–502 (2005)
Guyon, I., Elisseeff, A.: An Introduction to Variable and Feature Selection. Journal of Machine Learning Research 31, 157–1182 (2003)
LeCun, Y., Chopra, S., Hadsell, R., Huang, F.J., Ranzato, M.A.: A Tutorial on Energy-Based Learning. In: Bakir, et al. (eds.) Predicting Structured Outputs. MIT Press (2006)
Han, Y., Yu, L.: A Variance Reduction Framework for Stable Feature Selection. In: Proc. Int’l Conf. Data Mining, pp. 206–215 (2010)
Loscalzo, S., Yu, L., Ding, C.: Consensus Group Based Stable Feature Selection. In: Proc. of the 15th ACM SIGKDD Int’l Conf. on Knowledge Discovery and Data Mining (KDD 2009), pp. 567–576 (2009)
Saeys, Y., Abeel, T., Van de Peer, Y.: Robust Feature Selection Using Ensemble Feature Selection Techniques. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008, Part II. LNCS (LNAI), vol. 5212, pp. 313–325. Springer, Heidelberg (2008)
Weinberger, K.Q., Blitzer, J., Saul, L.K.: Distance Metric Learning for Large Margin Nearest Neighbor Classification. In: Advances in Neural Information Processing Systems, Cambridge MA, vol. 18 (2006)
Weinberger, K.Q., Saul, L.K.: Distance Metric Learning for Large Margin Nearest Neighbor Classification. Journal of Machine Learning Research 10, 207–214 (2009)
Li, Y., Lu, B.L.: Feature Selection Based on Loss Margin of Nearest Neighbor Classification. Pattern Recognition 42, 1914–1921 (2009)
Kira, K., Rendell, L.: A Practical Approach to Feature Selection. In: Proc. of Int’l Conf. on Machine Learning, pp. 249–256 (1992)
Robnik-Sikonja, M., Kononerko, I.: Theoretical and Empirical Analysis of Relief and ReliefF. Machine Learning, 23-69 (2003)
Merz, C.J., Murphy, P.M.: UCI repository of machine learning database (1996), http://www.ics.uci.edu/mlearn/MLRepository.html
Alon, U., Barkai, N., et al.: Broad Patterns of Gene Expression Revealed by Clustering Analysis of Tumor and Normal Colon Cancer Tissues Probed by Oligonucleotide Arrays. Proc. of the National Academy of Sciences of the United States of America 96, 6745–6750 (1999)
Chang, C.C., Lin, C.J.: LIBSVM: a Library for Support Vector Machines, http://www.csie.ntu.edu.tw/~cjlin/papers/libsvm.ps.gz
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Li, Y., Gao, SY. (2011). Energy-Based Feature Selection and Its Ensemble Version. In: Lu, BL., Zhang, L., Kwok, J. (eds) Neural Information Processing. ICONIP 2011. Lecture Notes in Computer Science, vol 7063. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-24958-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-642-24958-7_7
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-24957-0
Online ISBN: 978-3-642-24958-7
eBook Packages: Computer ScienceComputer Science (R0)