Dimension Reduction with Semi-supervised Pairwise Covariance-Preserving Projection

Abstract

Dimension reduction is critical in many areas of pattern classification and machine learning and many algorithms have been proposed. Pairwise Covariance-preserving Projection Method (PCPM) was an effective dimension reduction which maximizes the class discrimination and also preserves approximately the pairwise class covariances. A shortcoming of PCPM is that it can only be applied when all labels are given, thus a typical supervised method. Semi-supervised has attracted much attention in recent years since they can utilize both labeled and unlabeled data. In this paper, we extend PCPM to semi-supervised setting. The labeled data points are used to maximize the separability between different classes and the unlabeled data points are used to estimate the intrinsic geometric structure of the data. Specifically, we aim to learn a discriminant function which is as smooth as possible on the data manifold. The target optimization problem involved can be solved efficiently by eigenvalue decomposition. Experimental results on several datasets demonstrate the effectiveness of our method.