Experimental study of performance of pattern classifiers and the size of design samples

https://doi.org/10.1016/0167-8655(94)00099-OGet rights and content

Abstract

This paper presents results of simulation experiments concerning the two class pattern classification problem assuming a multivariate normal population. Both a quadratic and a linear discriminant function designed by using estimated covariance matrices and mean vectors have limited performance compared to the optimal Bayes decision. Two approximate estimators of the amount of degradation in the recognition rate were proposed by Raudys and Fukunaga, respectively. This paper presents the experimental evaluation of the goodness of these estimators. We show quantitatively how well those estimators work and also confirm that the modified classifier designed by using Stein's estimator achieves better performance than the conventional one.

References (7)

There are more references available in the full text version of this article.

Cited by (8)

  • Learning algorithms may perform worse with increasing training set size: Algorithm-data incompatibility

    2014, Computational Statistics and Data Analysis
    Citation Excerpt :

    One problem that concerns practitioners in all of the applications of learning is the size of the training dataset. Some theoretical results, simulations, and experience in the majority of applications indicate that the performance of a learning method improves as the training sample size increases (Yousef et al., 2004; Chan, 2003; Chan et al., 1998, 1999, 1997; Raudys, 1997; Raudys and Pikelis, 1980; Raudys and Jain, 1991; Takeshita and Toriwaki, 1995; Dobbin and Simon, 2007). Qualitatively speaking, performance is measured as the capability of the learning method to predict an unknown output from a new but known input.

  • Sample size issues in the choice between the best classifier and fusion by trainable combiners

    2014, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
  • Likelihood-ratio-based verification in high-dimensional spaces

    2014, IEEE Transactions on Pattern Analysis and Machine Intelligence
  • Verification under increasing dimensionality

    2010, Proceedings - International Conference on Pattern Recognition
View all citing articles on Scopus
1

A part of this research was supported by Grant-in-Aid for Scientific Research of the Ministry of Education, Science and Culture (05558035) and the Hori Information Science Promotion Foundation.

View full text