Handwritten Digit Recognition with Nonlinear Fisher Discriminant Analysis

Abstract

To generalize the Fisher Discriminant Analysis (FDA) algorithm to the case of discriminant functions belonging to a nonlinear, finite dimensional function space $\mathcal{F}$ (Nonlinear FDA or NFDA), it is sufficient to expand the input data by computing the output of a basis of $\mathcal{F}$ when applied to it [1,2,3,4]. The solution to NFDA can then be found like in the linear case by solving a generalized eigenvalue problem on the between- and within-classes covariance matrices (see e.g.[5]). The goal of NFDA is to find linear projections of the expanded data (i.e., nonlinear transformations of the original data) that minimize the variance within a class and maximize the variance between different classes. Such a representation is of course ideal to perform classification. The application of NFDA to pattern recognition is particularly appealing, because for a given input signal and a fixed function space it has no parameters and it is easy to implement and apply. Moreover, given C classes only C–1 projections are relevant [5]. As a consequence, the feature space is very small and the algorithm has low memory requirements and high speed during recognition.

This work has been supported by a grant from the Volkswagen Foundation.

An erratum to this chapter can be found at http://dx.doi.org/10.1007/11550907_163 .