Abstract
The paper presents a new method for selecting features that is suited for the instance-based classification. The selection is based on the ReliefF estimation of the quality of features in the orthogonal feature space obtained after PCA transformation, as well as on the interpretation of these weights as values proportional to the amount of explained concept changes. The user sets a threshold defining what percent of the whole concept variability the selected features should explain and only the first “stronger” features, which combine weights together exceed this threshold, are selected. During the classification phase the selected features are used along with their weights. The experiment results on 12 benchmark databases have shown the advantages of the proposed method in comparison with traditional ReliefF.
Keywords
This is a preview of subscription content, log in via an institution.
References
Bins, J., Draper, B.: Feature selection from huge feature sets. In: Proceedings of the Eighth IEEE International Conference on Computer Vision, vol. 2, pp. 159–165 (2001)
Chang, C.-C.: Generalized iterative RELIEF for supervised distance metric learning. Pattern Recogn. 43(8), 2971–2981 (2010)
Dhanabal, S., Chandramathi, S.: A review of various k-nearest neighbor query processing techniques. Intern. J. Comput. Appl. 31(7), 14–22 (2011)
Diamataras, K.I., Kung, S.J.: Principal Component Neural Networks. Theory and Applications. Wiley, New York (1996)
Florez-lopez, R.: Reviewing RELIEF and its extensions: a new approach for estimating attributes considering high-correlated features. In: Proceedings of IEEE International Conference on Data Mining, Maebashi, Japan, pp. 605–608 (2002)
Freitag, D., Caruana, R.: Greedy attribute selection. In: Proceedings of Eleven International Conference on Machine Learning, pp. 28–36 (1994)
Hall, M.A.: Correlation-based feature selection of discrete and numeric class machine learning. In: Proceedings of International Conference on Machine Learning (ICML-2000), San Francisco, CA, pp. 359–366. Morgan Kaufmann, San Francisco (2000)
Jolliffe, I.T.: Principal Component Analysis. Springer, New York (1986)
Kira, K., Rendell, L.A.: The feature selection problem: traditional methods and a new algorithm. In: Proceedings of AAAI 1992, San Jose, USA, pp. 129–134 (1992)
Kononenko, I.: Estimating attributes: analysis and extensions of RELIEF. In: Proceedings of European Conference on Machine Learning, Catania, Italy, vol. 182, pp. 171–182 (1994)
Ordonez, C., Mohanam, N., Garcia-Alvarado, C.: PCA for large data sets with parallel data summarization. Distrib. Parallel Databases 32(3), 377–403 (2014)
Pechenizkiy, M.: The impact of feature extraction on the performance of a classifier: kNN, Naïve Bayes and C4.5. In: Kégl, B., Lee, H.-H. (eds.) Canadian AI 2005. LNCS (LNAI), vol. 3501, pp. 268–279. Springer, Heidelberg (2005)
Robnik-Sikonja, M., Kononenko, I.: Theoretical and empirical analysis of ReliefF and RReliefF. Mach. Learn. J. 53, 23–69 (2003)
Sharma, A., Paliwala, K., Onwubolu, G.: Class-dependent PCA, MDC and LDA: a combined classifier for pattern classification. Pattern Recogn. 39, 1215–1229 (2006)
Strandjev, B., Agre, G.: On impact of PCA for solving classification tasks defined on facial images. Intern. J. Reason. Based Intell. Syst. 6(3/4), 85–92 (2014)
Sun, Y., Li, J.: Iterative RELIEF for feature weighting: algorithms, theories, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 1035–1051 (2007)
Tsymbal, A., Puuronen, S., Pechenizkiy, M., Baumgarten, M., Patterson, D.W.: Eigenvector-based feature extraction for classification. In: Proceedings of FLAIRS Conference, pp. 354–358 (2002)
Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cogn. Neurosci. 3(1), 71–86 (1991)
Vergara, J., Estevez, P.: A review of feature selection methods based on mutual information. Neural Comput. Appl. 24, 175–186 (2014)
Wettschereck, D., Aha, D.W., Mohri, T.: A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Artif. Intell. Rev. 11, 273–314 (1997)
Yang, J., Li, Y.-P.: Orthogonal relief algorithm for feature selection. In: Huang, D.-S., Li, K., Irwin, G.W. (eds.) ICIC 2006. LNCS, vol. 4113, pp. 227–234. Springer, Heidelberg (2006)
Zeng, X., Wang, Q., Zhang, C., Cai, H.: Feature selection based on ReliefF and PCA for underwater sound classification. In: Proceedings of the 3rd International Conference on Computer Science and Network Technology (ICCSNT), Dalian, pp. 442–445 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Agre, G., Dzhondzhorov, A. (2016). A Weighted Feature Selection Method for Instance-Based Classification. In: Dichev, C., Agre, G. (eds) Artificial Intelligence: Methodology, Systems, and Applications. AIMSA 2016. Lecture Notes in Computer Science(), vol 9883. Springer, Cham. https://doi.org/10.1007/978-3-319-44748-3_2
Download citation
DOI: https://doi.org/10.1007/978-3-319-44748-3_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-44747-6
Online ISBN: 978-3-319-44748-3
eBook Packages: Computer ScienceComputer Science (R0)