Abstract
The Pseudo Fisher Linear Discriminant (PFLD) based on a pseudo-inverse technique shows a peaking behaviour of the generalization error for training sample sizes that are about the feature size: with an increase in the training sample size the generalization error at first decreases reaching the minimum, then increases reaching the maximum at the point where the training sample size is equal to the data dimensionality and afterwards begins again to decrease. A number of ways exist to solve this problem. In this paper it is shown that noise injection by adding redundant features to the data also helps to improve the generalization error of this classifier for critical training sample sizes.
Chapter PDF
Keywords
Reference
K. Fukunaga, Introduction to Statistical Pattern Recognition. Academic Press, 400–407 (1990).
R.P.W. Duin, Small sample size generalization, Proceedings of 9th Scandinavian Conference on Image Analysis, Uppsala, Sweden, 957–964 (1995).
M. Skurichina and R.P.W. Duin, Stabilizing classifiers for very small sample sizes, Proceedings of ICPR, Vienna, Austria, 891–896 (1996).
M. Skurichina and R.P.W. Duin, Bagging for Linear Classifiers, Pattern Recognition, vol. 31, no. 7 (1998), in press.
Š. Raudys, M. Skurichina, T. Cibas and P. Gallinari, Optimal Regularization of Neural Networks and Ridge Estimates of the Covariance Matrix in Statistical Classification, In: Pattern Recognition and Image Analysis: Advances in Mathematical Theory and Applications (an Int. Journal of Russian Academy of Sciences), Vol. 5, No. 4, 1995, pp. 633–650.
H. Netten, I.T. Young, M. Prins, L.J. van Vliet, H.J. Tanke, J. Vrolijk, W. Sloos, Automation of Fluorescent dot counting in cell nuclei, Proceedings of the 12th Int. Conference on Pattern Recognition, Vol. 1, Jerusalem, 84–87 (1994).
A. Hoekstra, H. Netten and D. de Ridder, A neural network applied to spot counting, Proceedings of ACSI'96, the Second Annual Conference of the Advanced School for Computing and Imaging, Lommel, Belgium, 224–229 (1996).
R.A. Fisher, The Use of multiple measurements in taxonomic problems, Annals of Eugenics 7, no. 2 (1936).
R.A. Fisher, The precision of discriminant functions, Annals of Eugenics 10, no. 4 (1940).
R. Rao, On some problems arising of discrimination with multiple characters, Sankya 9, 343–365 (1949).
Š. Raudys and V. Pikelis, On dimensionality, sample size, classification error and complexity of classification algorithm in pattern recognition, IEEE Transaction on Pattern Analysis and Machine Intelligence PAMI-2, no. 3, 242–252 (1980).
Š. Raudys and R.P.W. Duin, On expected classification error of the Fisher linear classifier with pseudo-inverse covariance matrix, Pattern Recognition Letters (1998), in press.
C. Cortes and V. Vapnik, Support-vector networks, Machine Learning, Vol. 20, No. 3, pp. 273–297 (1995).
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 1998 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Skurichina, M., Duin, R.P.W. (1998). Regularization by adding redundant features. In: Amin, A., Dori, D., Pudil, P., Freeman, H. (eds) Advances in Pattern Recognition. SSPR /SPR 1998. Lecture Notes in Computer Science, vol 1451. Springer, Berlin, Heidelberg. https://doi.org/10.1007/BFb0033280
Download citation
DOI: https://doi.org/10.1007/BFb0033280
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-64858-1
Online ISBN: 978-3-540-68526-5
eBook Packages: Springer Book Archive
