Abstract
The k-nearest neighbor (k-NN) classification is a simple and effective classification approach. However, it suffers from over-sensitivity problem due to irrelevant and noisy features. In this paper, we propose an algorithm to improve the effectiveness of k-NN by combining these two approaches. Specifically, we select all relevant features firstly, and then assign a weight to each one. Experimental results show that our algorithm achieves the highest accuracy or near to the highest accuracy on all test datasets. It also achieves higher generalization accuracy compared with the well-known algorithms IB1-4 and C4.5.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
D.W. Aha, D. Kibler & M.K. Albert, “Instance-Based Learning Algorithms”, Machine Learning, Vol. 6, 1991,pp. 37–66.
D.W. Aha, “Tolerating Noisy, Irrelevant and Novel Attributes in Instance-Based Learning Algorithm”, International Journal of Man-Machine Studies, Vol. 36, No. 1, 1992, pp. 267–287.
D.W. Aha & R.L. Bankert, “Feature Selection for Case-Based Classification of Cloud Types: An Empirical Comparison”, In D.W. Aha (Ed.) Case-Based Reasoning: Papers from the 1994 Worksho ((TR WS-94-10), Menlo Park, CA: AAAI press.
D.W. Aha, “Feature Weighting for Lazy Learning Algorithms”, In H. Liu & H. Motoda (Eds.) Feature Extraction, Construction and Selection: A Data Mining Perspective, Norwell MA: Kluwer, 1998.
N.F. Ayan, “Using Information Gain as Feature Weight”, 8th Turkish Symposium on Artificial Intelligence and Neural Networks (TAINN’99), Istanbul, Turkey, June 1999.
Y. Bao, X. Du & N. Ishii, “Using Rough Sets and Class Mutual Information for Feature Selection”, 6th International Conference on Soft Computing, IIZUKA2000, pp. 452–458.
R. Battiti, “Using Mutual Information for Selection Features in Supervised Neural Net Learning,” IEEE Trans. On Neural Networks, 5(4), 1994, pp. 537–550.
C. Cardie, “A Cognitive Bias Approach to Feature Selection and Weighting for Case-Based Learners”, Machine Learning, Vol. 41, 2000, pp. 85–116.
T.M. Cover & P.E. Hart, “Nearest Neighbor Pattern Classification”, IEEE Transactions on Information Theory, Vol. 13, No. 1, 1967, pp. 21–27.
T.M. Cover & J.A. Thomas, Elements of Information Theory, 1991.
R. Kohavi, P. Langley & Y. Yun, “The Utility of Feature Weighting in Nearest Neighbor Algorithms”, ECML-97 (poster).
K.C. Lee, “A Technique of Dynamic Feature Selection Using the Feature Group Mutual Information”, Third Pacific Asia Conference, PAKDD-99, LNAI 1574, 1999, pp. 138–142.
C. J. Merz & P.M. Murphy, 1998, UCI Repository of Machine Learning Databases, Irvine, CA: University of California Irvine, Department of Information and Computer Science, Internet: http://www.ics.uci.edu/ mlearn/MLRepository.html.
Z. Pawlak, ROUGH SETS (Theoretical Aspects of Reasoning about Data), Kluwer (1991).
J.R. Quinlan, “Induction of Decision Trees”. Machine Learning 1(1), 1986, pp. 81–106.
J.R. Quinlan, C4.5: Programs for Machine Learning, Morgan Kaufmann, 1993.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2002 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bao, Y., Du, X., Ishii, N. (2002). Combining Feature Selection with Feature Weighting for k-NN Classifier. In: Yin, H., Allinson, N., Freeman, R., Keane, J., Hubbard, S. (eds) Intelligent Data Engineering and Automated Learning — IDEAL 2002. IDEAL 2002. Lecture Notes in Computer Science, vol 2412. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-45675-9_69
Download citation
DOI: https://doi.org/10.1007/3-540-45675-9_69
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-44025-3
Online ISBN: 978-3-540-45675-9
eBook Packages: Springer Book Archive