Abstract
Feature selection is one of the most important data preprocessing techniques in data mining and machine learning. A new feature selection method based on density peak clustering is proposed. The new method applies an information distance between features as clustering distance metric, and uses the density peak clustering method for feature clustering. The representative feature of each cluster is selected to generate the final result. The method can avoid selecting the irrelevant representative feature from one cluster, where most features are irrelevant to class label. The comparison experiments on ten datasets show that the feature selection results of the proposed method exhibit improved classification accuracies for different classifiers.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Avrim, L.B., Langley, P.: Selection of relevant features and examples in machine learning. Artif. Intell. 97(1), 245–271 (1997)
Liu, H., Motoda, H.: Feature Selection for Knowledge Discovery and Data Mining. Kluwer Academic Publishers, Boston (1998)
Peng, H., Long, F., Ding, C.: Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans. Pattern Anal. Mach. Intell. 27(8), 1226–1238 (2005)
Fleuret, F.: Fast binary feature selection with conditional mutual information. J. Mach. Learn. Res. 5, 1531–1555 (2004)
Hall, M.A.: Correlation-based feature selection for discrete and numeric class machine learning. In: Proceedings of the 7th International Conference on Machine Learning, pp. 359–366 (2000)
Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. J. Mach. Learn. Res. 5, 1205–1224 (2004)
Mitra, P., Murthy, C.: Unsupervised feature selection using similarity. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 301–312 (2002)
Ienco, D., Meo, R.: Exploration and reduction of the feature space by hierarchical clustering. In: Proceedings of the 2008 SIAM Conference on Data Mining, Atlanta, Georgia, USA, pp. 577–587 (2008)
Witten, D., Tibshirni, R.: A framework for feature selection in clustering. J. Am. Stat. Associ. 105, 713–726 (2010)
Liu, H., Wu, X., Zhang, S.: Feature selection using hierarchical feature clustering. In: Proceedings of the 20th ACM International Conference on Information and Knowledge Management, Glasgow, United Kingdom, pp. 979–984 (2011)
Zhao, X., Deng, W., Shi, Y.: Feature selection with attributes clustering by maximal information coefficient. Procedia Comput. Sci. 17(2), 70–79 (2013)
Bandyopadhyay, S., Bhadra, T., Mitra, P., et al.: Integration of dense subgraph finding with feature clustering for unsupervised feature selection. Pattern Recogn. Lett. 40(1), 104–112 (2014)
Au, W.H., Chan, K.C., Wong, A.K., Wang, Y.: Attribute clustering for grouping, selection, and classification of gene expression data. IEEE/ACM Trans. Comput. Biol. Bioinf. 2(2), 83–101 (2005)
Song, Q., Ni, J., Wang, G.: A fast clustering-based feature subset selection algorithm for high-dimensional data. IEEE Trans. Knowl. Data Eng. 25(1), 1–14 (2013)
Liu, Q., Zhang, J., Xiao J., Zhu, H., Zhao, Q.: A supervised feature selection algorithm through minimum spanning tree clustering. In: IEEE 26th International Conference on Tools with Artificial Intelligence, pp. 264–271 (2014)
Meila, M.: Comparing clusterings - an information based distance. J. Multivar. Anal. 98(5), 873–895 (2007)
Vinh, N.X., Epps, J., Bailey, J.: Information theoretic measures for clusterings comparison: variants, properties, normalization and correction for chance. J. Mach. Learn. Res. 11(10), 2837–2854 (2010)
Rodriguez, A., Laio, A.: Clustering by fast search and find of density peaks. Science 344(6191), 1492–1496 (2014)
Fayyad, U.M., Irani, K.B.: Multi-interval discretization of continuous-valued attributes for classification learning. In: Proceedings of the IJCAI, pp. 1022–1029 (1993)
Yang, H.H., Moody, J.E.: Data visualization and feature selection: new algorithms for nongaussian data. In: Proceedings of the NIPS, pp. 687–693 (1999)
Kononenko, I.: Estimating attributes: analysis and extensions of RELIEF. Mach. Learn. 14, 171–182 (1994)
Sotoca, J.M., Pla, F.: Supervised feature selection by clustering using conditional mutual information-based distances. Pattern Recogn. 43(6), 2068–2081 (2010)
Acknowledgment
The authors would like to acknowledge the assistance provided by National Natural Science Foundation of China (Grant no. 61572180, no. 61472467 and no. 61672011).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Cai, J., Chao, S., Yang, S., Wang, S., Luo, J. (2017). Feature Selection Based on Density Peak Clustering Using Information Distance Measure. In: Huang, DS., Jo, KH., Figueroa-García, J. (eds) Intelligent Computing Theories and Application. ICIC 2017. Lecture Notes in Computer Science(), vol 10362. Springer, Cham. https://doi.org/10.1007/978-3-319-63312-1_11
Download citation
DOI: https://doi.org/10.1007/978-3-319-63312-1_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-63311-4
Online ISBN: 978-3-319-63312-1
eBook Packages: Computer ScienceComputer Science (R0)