Abstract
Nearest Neighbor (NN) rule is one of the simplest and most important methods in pattern recognition. In this paper, we propose a kernel difference-weighted k-nearest neighbor method (KDF-WKNN) for pattern classification. The proposed method defines the weighted KNN rule as a constrained optimization problem, and then we propose an efficient solution to compute the weights of different nearest neighbors. Unlike distance-weighted KNN which assigns different weights to the nearest neighbors according to the distance to the unclassified sample, KDF-WKNN weights the nearest neighbors by using both the norm and correlation of the differences between the unclassified sample and its nearest neighbors. Our experimental results indicate that KDF-WKNN is better than the original KNN and distance-weighted KNN, and is comparable to some state-of-the-art methods in terms of classification accuracy.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Bailey, T., Jain, A.K.: A Note on Distance-Weighted K-Nearest Neighbor Rules. IEEE Trans. Systems, Man, and Cybernetics 8, 311–313 (1978)
Demšar, J.: Statistical Comparisons of Classifiers over Multiple Data Sets. Journal of Machine Learning Research 7, 1–30 (2006)
Domeniconi, C., Peng, J., Gunopulos, D.: Locally Adaptive Metric Nearest Neighbor Classification. IEEE Trans. PAMI 24, 1281–1285 (2002)
Dudani, S.A.: The Distance-Weighted K-Nearest-Neighbor Rule. IEEE Trans. Systems, Man, and Cybernetics 6, 325–327 (1976)
Fukunaga, K., Flick, T.E.: An Optimal Global Nearest Neighbor Metric. IEEE Trans. PAMI 6, 314–318 (1984)
Hastie, T., Tibshirani, R.: Discriminant Adaptive Nearest Neighbor Classification. IEEE Trans. PAMI 18, 607–616 (1996)
Keller, J.M., Gray, M.R., Givens Jr., J.A.: A Fuzzy K-Nearest Neighbor Algorithm. IEEE Trans. Systems, Man, and Cybernetics 15, 580–585 (1985)
Macleod, J.E.S., Luk, A., Titterington, D.M.: A Re-examination of the Distance-Weighted K-Nearest Neighbor Classification Rule. IEEE Trans. SMC 17, 689–696 (1987)
Müller, K.R., Mika, S., Rätsch, G., Tsuda, K., Schölkopf, B.: An Introduction to Kernel-based Learning Algorithms. IEEE Trans. Neural Networks 12, 181–202 (2001)
Paredes, R., Vidal, E.: Learning Weighted Metrics to Minimizing Nearest-Neighbor Classification Error. IEEE Trans. PAMI 28, 1100–1110 (2006)
Ricci, F., Avesani, P.: Data Compression and Local Metrics for Nearest Neighbor Classification. IEEE Trans. PAMI 21, 380–384 (1999)
Short, R.D., Fukunaga, K.: The Optimal Distance Measure for Nearest Neighbor Classification. IEEE Trans. Information Theory 27, 622–627 (1981)
Toh, K.A., Tran, Q.L., Srinivasan, D.: Benchmarking a Reduced Multivariate Polynormial Pattern Classifier. IEEE Trans. PAMI 26, 740–755 (2004)
Wang, H.: Nearest Neighbors by Neighborhood Counting. IEEE Trans. PAMI 28, 942–953 (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zuo, W., Wang, K., Zhang, H., Zhang, D. (2007). Kernel Difference-Weighted k-Nearest Neighbors Classification. In: Huang, DS., Heutte, L., Loog, M. (eds) Advanced Intelligent Computing Theories and Applications. With Aspects of Artificial Intelligence. ICIC 2007. Lecture Notes in Computer Science(), vol 4682. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74205-0_89
Download citation
DOI: https://doi.org/10.1007/978-3-540-74205-0_89
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74201-2
Online ISBN: 978-3-540-74205-0
eBook Packages: Computer ScienceComputer Science (R0)