Abstract
In this paper, the problem of classifying an unseen pattern on the basis of its nearest neighbors in a recorded data set is addressed from the point of view of Dempster-Shafer theory. Each neighbor of a sample to be classified is considered as an item of evidence that supports certain hypotheses regarding the class membership of that pattern. The degree of support is defined as a function of the distance between the two vectors. The evidence of the k nearest neighbors is then pooled by means of Dempster’s rule of combination. This approach provides a global treatment of such issues as ambiguity and distance rejection, and imperfect knowledge regarding the class membership of training patterns. The effectiveness of this classification scheme as compared to the voting and distance-weighted k-NN procedures is demonstrated using several sets of simulated and real-world data
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Bibliography
T. Baily and A. K. Jain, “A note on distance-weighted k-nearest neighbor rules,” IEEE Trans. Syst. Man Cyber., vol. 8, no. 4, pp. 311–313, 1978.
W. F. Caselton and W. Luo, “Decision making with imprecise probabilities: Dempster-Shafer theory and application,” Water Resources Research, vol. 28, no. 2, pp. 3071–3081, 1992.
C. K. Chow, “On optimum recognition error and reject tradeoff,” IEEE Trans. Inform. Theory, vol. IT-16, pp. 41–46, 1970.
T. M. Cover and P. E. Hart, “Nearest neighbor pattern classification,” IEEE Trans. Inform. Theory, vol. IT-13, no. 1, pp. 21–27, 1967.
B. V. Dasarathy, “Nosing around the neighborhood: A new system structure and classification rule for recognition in partially exposed environments,” IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, no. 1, pp. 67–71, 1980.
B. V. Dasarathy, “Nearest neighbor norms: NN pattern classification techniques,” IEEE Computer Society Press, Los Alamitos, CA, 1991.
A. P. Dempster and A. Kong, “Comment,” Stat. Sci., vol. 2, no. 1, pp. 32–36, 1987.
D. H. Deterding, “Speaker normalization for automatic speech recognition,” Ph.D. thesis, University of Cambridge, 1989.
B Dubuisson and M. Masson, “A statistical decision rule with incomplete knowledge about classes,” Pattern Recognition, vol. 26, no. 1, pp. 155–165, 1993.
S. A. Dudani, “The distance-weighted k-nearest-neighbor rule,” IEEE Trans. Syst. Man Cyber., vol. 6, pp. 325–327, 1976.
E. Fix and J. L. Hodges, “Discriminatory analysis, nonparametric discrimination: Consistency properties,” Technical Report 4, USAF School of Aviation Medicine, Randolph Field, TX, 1951.
M. E. Hellman, “The nearest neighbor classification rule with a reject option,” IEEE Trans. Syst. Man Cyber., vol. 3, pp. 179–185, 1970.
A. Jozwik, “A learning scheme for a fuzzy k-NN rule,” Pattern Recognition Letters, vol. 1, pp. 287–289, 1983.
J. M. Keller, M. R. Gray, and J. A. Givens, “A fuzzy k-NN neighbor algorithm,” IEEE Trans. Syst. Man Cyber., vol. 15, no. 4, pp. 580–585, 1985.
J. E. Macleod, A. Luk, and D. M. Titterington, “A re-examination of the distance-weighted k-nearest neighbor classification rule,” IEEE Trans. Syst. Man Cyber., vol. 17, no. 4, pp. 689–696, 1987.
R. L. Morin and D. E. Raeside, “A reappraisal of distance-weighted k-nearest-neighbor classification for pattern recognition with missing data,” IEEE Trans. Syst. Man Cyber., vol. 11, no. 2, pp. 241–243, 1981.
P. M. Murphy and D. W. Aha, “UCI Repository of machine learning databases [Machine-readable data repository],” University of California, Department of Information and Computer Science., Irvine, CA, 1994.
A. J. Robinson, “Dynamic error propagation networks,” Ph.D. thesis, Cambridge University Engineering Department, 1989.
G. Shafer, A Mathematical Theory of Evidence. Princeton, NJ: Princeton University Press, 1976.
V. G. Sigillito, S. P. Wing. L. V. Hutton, and K. B. Baker, “Classification of radar returns from the ionosphere using neural networks,” in Johns Hopkins APL Technical Digest, vol. 10, pp. 262–266, 1989.
B. Tessem, “Approximations for efficient computation in the theory of evidence,” Artificial Intelligence, vol. 61, pp. 315–329, 1993.
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Denœux, T. (2008). A k-Nearest Neighbor Classification Rule Based on Dempster-Shafer Theory. In: Yager, R.R., Liu, L. (eds) Classic Works of the Dempster-Shafer Theory of Belief Functions. Studies in Fuzziness and Soft Computing, vol 219. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-44792-4_29
Download citation
DOI: https://doi.org/10.1007/978-3-540-44792-4_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-25381-5
Online ISBN: 978-3-540-44792-4
eBook Packages: EngineeringEngineering (R0)