On Solving the Small Sample Size Problem for Marginal Fisher Analysis

Abstract

Marginal Fisher Analysis (MFA) was introduced to remedy some of the shortcomings of the Fisher Discriminant Analysis (FDA). It performs local discrimination between classes. Whenever the training data set is small, MFA cannot directly be used with the original high-dimensional samples. This is referred to as the small sample size (SSS) phenomenon that happens whenever the feature dimension is higher than the number of examples. The classic remedy was using the projection of the raw data (e.g., using (PCA)). This paper introduces two regularization schemes that overcome the singularity and near singularity of the locality preserving scatters. The first scheme uses ridge regression regularization. The second scheme uses matrix exponential and introduces an implicit distance diffusion mapping. The experiments are conducted on four face data sets. These experiments demonstrate that the introduced schemes can enhance the performance of the MFA framework much better than the widely used PCA based regularization.