Abstract
Research effort has recently focused on designing feature weighting clustering algorithms. These algorithms automatically calculate the weight of each feature, representing their degree of relevance, in a data set. However, since most of these evaluate one feature at a time they may have difficulties to cluster data sets containing features with similar information. If a group of features contain the same relevant information, these clustering algorithms set high weights to each feature in this group, instead of removing some because of their redundant nature. This paper introduces an unsupervised feature selection method that can be used in the data pre-processing step to reduce the number of redundant features in a data set. This method clusters similar features together and then selects a subset of representative features for each cluster. This selection is based on the maximum information compression index between each feature and its respective cluster centroid. We present an empirical validation for our method by comparing it with a popular unsupervised feature selection on three EEG data sets. We find that our method selects features that produce better cluster recovery, without the need for an extra user-defined parameter.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
de Amorim, R.C.: An empirical evaluation of different initializations on the number of k-means iterations. Lect. Notes Comput. Sci. 7629, 15–26 (2013)
de Amorim, R.C.: Feature relevance in Ward’s hierarchical clustering using the Lp norm. J. Classif. 32(1) (to appear in 2015)
de Amorim, R.C., Komisarczuk, P.: On initializations for the Minkowski weighted k-means. Lect. Notes Comput. Sci. 7619, 45–55 (2012)
de Amorim, R.C., Mirkin, B.: Minkowski metric, feature weighting and anomalous cluster initializing in k-means clustering. Pattern Recogn. 45(3), 1061–1075 (2012)
de Amorim, R.C., Mirkin, B.: Removing redundant features via clustering: preliminary results in mental task separation. In: Proceedings of the 8th International Conference on Knowledge, Information and Creativity Support Systems (KICSS), November, pp. 7–9. Krakow, Poland (2013)
de Amorim, R.C., Mirkin, B., Gan, J.Q.: A method for classifying mental tasks in the space of EEG transforms. Technical report, Technical Report BBKS-10-01, Birkbeck University of London, London (2010)
de Amorim, R.C., Mirkin, B., Gan, J.Q.: Anomalous pattern based clustering of mental tasks with subject independent learning-some preliminary results. Artif. Intell. Res. 1(1), 46–54 (2012)
Ball, G.H., Hall, D.J.: A clustering technique for summarizing multivariate data. Behav. Sci. 12(2), 153–155 (1967)
Bezdek, J.C.: Pattern Recognition with Fuzzy Objective Function Algorithms. Kluwer Academic Publishers, Norwell, MA (1981)
Celebi, M.E., Kingravi, H.A., Vela, P.A.: A comparative study of efficient initialization methods for the k-means clustering algorithm. Expert Syst. Appl. 40(1), 200–210 (2013)
Chan, E.Y., Ching, W.K., Ng, M.K., Huang, J.Z.: An optimization algorithm for clustering using weighted dissimilarity measures. Pattern Recogn. 37(5), 943–952 (2004)
Chiang, M.M.T., Mirkin, B.: Intelligent choice of the number of clusters in k-means clustering: an experimental study with different cluster spreads. J. Classif. 27(1), 3–40 (2010)
Chiappa, S., Bengio, S.: HMM and IOHMM modeling of EEG rhythms for asynchronous BCI systems. In: European Symposium on Artificial Neural Networks, ESANN, pp. 193–204 (2004)
De Soete, G.: Optimal variable weighting for ultrametric and additive tree clustering. Qual. Quant. 20(2–3), 169–180 (1986)
De Soete, G.: OVWTRE: a program for optimal variable weighting for ultrametric and additive tree fitting. J. Classif. 5(1), 101–104 (1988)
DeSarbo, W.S., Carroll, J.D., Linda, C.A., Green, P.E.: Synthesized clustering: a method for amalgamating alternative clustering bases with differential weighting of variables. Psychometrika 49(1), 57–78 (1984)
Frigui, H., Nasraoui, O.: Unsupervised learning of prototypes and attribute weights. Pattern Recogn. 37(3), 567–581 (2004)
Gan, J.Q.: Self-adapting BCI based on unsupervised learning. In: 3rd International Workshop on Brain-Computer Interfaces, pp. 50–51 (2006)
Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)
Huang, J.Z., Ng, M.K., Rong, H., Li, Z.: Automated variable weighting in k-means type clustering. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 657–668 (2005)
Huang, J.Z., Xu, J., Ng, M., Ye, Y.: Weighting method for feature selection in k-means. In: Computational Methods of Feature Selection, pp. 193–209 (2008)
Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recogn. Lett. 31(8), 651–666 (2010)
MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, vol. 1, pp. 281–297. California, USA (1967)
Maitra, R., Peterson, A.D., Ghosh, A.P.: A systematic evaluation of different methods for initializing the k-means clustering algorithm. Trans. Knowl. Data Eng. 522–537 (2010)
Makarenkov, V., Legendre, P.: Optimal variable weighting for ultrametric and additive trees and k-means partitioning: methods and software. J. Classif. 18(2), 245–271 (2001)
Millan, J., Mouriño, J.: Asynchronous BCI and local neural classifiers: an overview of the adaptive brain interface project. IEEE Trans. Neural Syst. Rehabil. Eng. 11(2), 159–161 (2003)
Milligan, G.W., Cooper, M.C.: A study of standardization of variables in cluster analysis. J. Classif. 5(2), 181–204 (1988)
Mirkin, B.: Clustering for Data Mining: A Data Recovery Approach, vol. 3. CRC Press (2005)
Mitra, P., Murthy, C.A., Pal, S.K.: Unsupervised feature selection using feature similarity. IEEE Trans. Pattern Anal. Mach. Intell. 24(3), 301–312 (2002)
Pena, J.M., Lozano, J.A., Larranaga, P.: An empirical comparison of four initialization methods for the k-means algorithm. Pattern Recogn. Lett. 20(10), 1027–1040 (1999)
Steinbach, M., Karypis, G., Kumar, V.: A comparison of document clustering techniques. In: KDD Workshop on Text Mining, pp. 525–526. Boston (2000)
Steinley, D.: Standardizing variables in k-means clustering. In: Classification, Clustering, and Data Mining Applications, pp. 53–60. Springer (2004)
Steinley, D., Brusco, M.J.: Initializing k-means batch clustering: a critical evaluation of several techniques. J. Classif. 24(1), 99–121 (2007)
Svetlova, L., Mirkin, B., Lei, H.: MFWK-Means: Minkowski metric fuzzy weighted k-means for high dimensional data clustering. In: IEEE 14th International Conference on Information Reuse and Integration (IRI), pp. 692–699. IEEE (2013)
Tibshirani, R., Walther, G., Hastie, T.: Estimating the number of clusters in a data set via the gap statistic. J. R. Stat. Soc.: Ser. B (Stat. Methodol.) 63(2), 411–423 (2001)
Xu, R., Wunsch, D.: Survey of clustering algorithms. IEEE Trans. Neural Netw. 16(3), 645–678 (2005)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
de Amorim, R.C., Mirkin, B. (2016). A Clustering-Based Approach to Reduce Feature Redundancy. In: Skulimowski, A., Kacprzyk, J. (eds) Knowledge, Information and Creativity Support Systems: Recent Trends, Advances and Solutions. Advances in Intelligent Systems and Computing, vol 364. Springer, Cham. https://doi.org/10.1007/978-3-319-19090-7_35
Download citation
DOI: https://doi.org/10.1007/978-3-319-19090-7_35
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19089-1
Online ISBN: 978-3-319-19090-7
eBook Packages: Computer ScienceComputer Science (R0)