A Fast Fixed-Point Algorithm for Two-Class Discriminative Feature Extraction

Abstract

We propose a fast fixed-point algorithm to improve the Relevant Component Analysis (RCA) in two-class cases. Using an objective function that maximizes the predictive information, our method is able to extract more than one discriminative component of data for two-class problems, which cannot be accomplished by classical Fisher’s discriminant analysis. After prewhitening the data, we apply Newton’s optimization method which automatically chooses the learning rate in the iterative training of each component. The convergence of the iterative learning is quadratic, i.e. much faster than the linear optimization by gradient methods. Empirical tests presented in the paper show that feature extraction using the new method resembles RCA for low-dimensional ionosphere data and significantly outperforms the latter in efficiency for high-dimensional facial image data.

Supported by the Academy of Finland in the projects Neural methods in information retrieval based on automatic content analysis and relevance feedback and Finnish Centre of Excellence in Adaptive Informatics Research.