Training sets and a priori probabilities with the nearest neighbour method of pattern recognition

doi:10.1016/0167-8655(88)90017-7

Pattern Recognition Letters

Volume 8, Issue 1, July 1988, Pages 11-13

https://doi.org/10.1016/0167-8655(88)90017-7 Get rights and content

Abstract

This paper considers whether the nearest neighbour (NN) classifier takes proper account of a priori class probabilities. If the frequencies with which the different classes arise naturally in the whole population of patterns are retained during training, it is found that a priori probabilities are automatically incorporated: this will not be so if (for example) equal numbers of training patterns are selected from each class.

References (7)

R.O. Duda et al.
Pattern Classification and Scene Analysis
(1973)
T.M. Cover et al.
Nearest neighbour pattern classification
IEEE Trans. Inform. Theory
(1967)
P.A. Devijver et al.
Pattern Recognition: a Statistical Approach
(1982)

There are more references available in the full text version of this article.

Cited by (16)

Analyzing the oversampling of different classes and types of examples in multi-class imbalanced datasets
2016, Pattern Recognition
Citation Excerpt :
Because we mentioned that the random undersampling is not recommended, therefore most of the developed undersampling approaches are usually based on local neighborhood analysis and depend on the type of imbalance distribution, i.e., it is so-called overlapped, noisy or safe regions. It is worth mentioning ENN method (Edited Nearest Neighbour) [39], which labels a majority example as unreliable and discards it if there are no more majority class representatives among its three nearest neighbors. Other methods remove the examples from the majority class, which belong to homogeneous, safe regions and are far from the decision border, because they seem to be not so important for learning process [40,41].
Canonical machine learning algorithms assume that the number of objects in the considered classes are roughly similar. However, in many real-life situations the distribution of examples is skewed since the examples of some of the classes appear much more frequently. This poses a difficulty to learning algorithms, as they will be biased towards the majority classes. In recent years many solutions have been proposed to tackle imbalanced classification, yet they mainly concentrate on binary scenarios. Multi-class imbalanced problems are far more difficult as the relationships between the classes are no longer straightforward. Additionally, one should analyze not only the imbalance ratio but also the characteristics of the objects within each class. In this paper we present a study on oversampling for multi-class imbalanced datasets that focuses on the analysis of the class characteristics. We detect subsets of specific examples in each class and fix the oversampling for each of them independently. Thus, we are able to use information about the class structure and boost the more difficult and important objects. We carry an extensive experimental analysis, which is backed-up with statistical analysis, in order to check when the preprocessing of some types of examples within a class may improve the indiscriminate preprocessing of all the examples in all the classes. The results obtained show that oversampling concrete types of examples may lead to a significant improvement over standard multi-class preprocessing that do not consider the importance of example types.
Possibilistic instance-based learning
2003, Artificial Intelligence
A method of instance-based learning is introduced which makes use of possibility theory and fuzzy sets. Particularly, a possibilistic version of the similarity-guided extrapolation principle underlying the instance-based learning paradigm is proposed. This version is compared to the commonly used probabilistic approach from a methodological point of view. Moreover, aspects of knowledge representation such as the modeling of uncertainty are discussed. Taking the possibilistic extrapolation principle as a point of departure, an instance-based learning procedure is outlined which includes the handling of incomplete information, methods for reducing storage requirements and the adaptation of the influence of stored cases according to their typicality. First theoretical and experimental results showing the efficiency of possibilistic instance-based learning are presented as well.
Non-parametric probability estimation for HMM-based automatic speech recognition
2003, Computer Speech and Language
During the last decade, the most significant advances in the field of continuous speech recognition (CSR) have arisen from the use of hidden Markov models (HMM) for acoustic modeling. These models address one of the major issues for CSR: simultaneous modeling of temporal and frequency distortions in the speech signal. In the HMM, the temporal dimension is managed through an oriented states graph, each state accounting for the local frequency distortions through a probability density function. In this study, improvement of the HMM performance is expected from the introduction of a very effective non-parametric probability density function estimate: the k-nearest neighbors (k-nn) estimate.
First, experiments on a short-term speech spectrum identification task are performed to compare the k-nn estimate and the widespread estimate based on mixtures of Gaussian functions. Then adaptations implied by the integration of the k-nn estimate in an HMM-based recognition system are developed. An optimal training protocol is obtained based on the introduction of the membership coefficients in the HMM parameters. The membership coefficients measure the degree of association between a reference acoustic vector and a HMM state. The training procedure uses the expectation-maximization (EM) algorithm applied to the membership coefficient estimation. Its convergence is shown according to the maximum likelihood criterion. This study leads to the development of a baseline k-nn/HMM recognition system which is evaluated on the TIMIT speech database. Further improvements of the k-nn/HMM system are finally sought through the introduction of a temporal information into the representation space (delta coefficients) and the adaptation of the references (mainly, gender modeling and contextual modeling).
Improved multi-class classification approach for imbalanced big data on spark
2023, Journal of Supercomputing
Multi-class imbalanced data oversampling for vertebral column pathologies classification
2018, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Computer Vision: Principles, Algorithms, Applications, Learning: Fifth Edition
2017, Computer Vision: Principles, Algorithms, Applications, Learning: Fifth Edition

View all citing articles on Scopus

View full text

Training sets and a priori probabilities with the nearest neighbour method of pattern recognition

Abstract

Pattern Classification and Scene Analysis

Nearest neighbour pattern classification

IEEE Trans. Inform. Theory

Pattern Recognition: a Statistical Approach