Abstract
To generalize the Fisher Discriminant Analysis (FDA) algorithm to the case of discriminant functions belonging to a nonlinear, finite dimensional function space \(\mathcal{F}\) (Nonlinear FDA or NFDA), it is sufficient to expand the input data by computing the output of a basis of \(\mathcal{F}\) when applied to it [1,2,3,4]. The solution to NFDA can then be found like in the linear case by solving a generalized eigenvalue problem on the between- and within-classes covariance matrices (see e.g.[5]). The goal of NFDA is to find linear projections of the expanded data (i.e., nonlinear transformations of the original data) that minimize the variance within a class and maximize the variance between different classes. Such a representation is of course ideal to perform classification. The application of NFDA to pattern recognition is particularly appealing, because for a given input signal and a fixed function space it has no parameters and it is easy to implement and apply. Moreover, given C classes only Cā1 projections are relevant [5]. As a consequence, the feature space is very small and the algorithm has low memory requirements and high speed during recognition.
This work has been supported by a grant from the Volkswagen Foundation.
An erratum to this chapter can be found at http://dx.doi.org/10.1007/11550907_163 .
Similar content being viewed by others
References
Mika, S., RƤtsch, G., Weston, J., Schƶlkopf, B., MĆ¼ller, K.R.: Fisher discriminant analysis with kernels. In: Hu, Y.H., Larsen, J., Wilson, E., Douglas, S. (eds.) Proceedings of the IEEE Signal Processing Society Workshop. Neural Networks for Signal Processing, vol. IX, pp. 41ā48 (1999)
Baudat, G., Anouar, F.: Generalized discriminant analysis using a kernel approach. Neural Computation 12, 2385ā2404 (2000)
Mika, S., RƤtsch, G., Weston, J., Schƶlkopf, B., Smola, A., MĆ¼ller, K.R.: Invariant feature extraction and classification in kernel spaces. In: Solla, S., Leen, T., MĆ¼ller, K.R. (eds.) Advances in Neural Information Processing Systems, vol. 12, pp. 526ā532 (2000)
Mika, S., Smola, A., Schƶlkopf, B.: An improved training algorithm for kernel Fisher discriminants. In: Jaakkola, T., Richardson, T. (eds.): Proceedings AISTATS 2001, pp. 98ā104 (2001)
Bishop, C.M.: Neural Networks for Pattern Recognition. Oxford University Press, Oxford (1995)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278ā2324 (1998)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
Ā© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Berkes, P. (2005). Handwritten Digit Recognition with Nonlinear Fisher Discriminant Analysis. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds) Artificial Neural Networks: Formal Models and Their Applications ā ICANN 2005. ICANN 2005. Lecture Notes in Computer Science, vol 3697. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11550907_45
Download citation
DOI: https://doi.org/10.1007/11550907_45
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28755-1
Online ISBN: 978-3-540-28756-8
eBook Packages: Computer ScienceComputer Science (R0)