Robust 2DLDA based on correntropy

Abstract

To further improve the robustness of two-dimensional LDA (2DLDA) methods against outliers, this paper proposes a new robust 2DLDA version which obtains the optimal projection transformation by maximizing the correntropy-based within-class similarity and maintaining the global dispersity simultaneously. The objective problem of the proposed method can be solved by an iterative optimization algorithm which is proved to converge at a local maximum point. The experimental results on FERET face database, PolyU palmprint database and Binary Alphadigits database illustrate that the proposed method outperforms three conventional 2DLDA methods when there are outliers.

Introduction

Principal Component Analysis (PCA) [1] and Linear Discriminant Analysis (LDA) [2] are two well-known dimensionality reduction techniques for feature extraction, which have played a significant role in the fields of pattern recognition, machine learning and computer vision. PCA is an unsupervised technique and seeks the optimal projection transformation by maximizing the variance of training data in low-dimensional feature space. LDA is a supervised method and obtains the maximum class discrimination by means of maximizing the within-class similarity and the between-class dissimilarity simultaneously. LDA has been widely applied in many fields such as face recognition [3], hepatitis diagnosis [4], human action recognition [5] and motor bearing fault diagnosis [6]. Classic LDA extracts the low-dimensional feature just from the vector-based (1D) training data, which leads matrix-based (2D) data, such as images, to be transformed to vectors. Obviously, this transformation ignores the local spatial structure which is useful for expressing the intrinsic feature of image data. The multidimensional extension can improve the adaption of the 1D model [7]. Therefore, 2DLDA was proposed to extract features from the image matrices [8].

However, Frobenius-norm 2DLDA (2DLDA-L2) is sensitive to outliers because its similarity/dissimilarity metric adopts Frobenius-norm distance which magnifies the influence of the outlying data due to the square operation. In the last several years, L1-norm was popularly applied to polish up the robustness of PCA against outliers [9], [10], [11], [12]. Motivated by the essence of PCA based on rotational invariant L1-norm (PCA-R1) [9], LDA-R1 and 2DLDA-R1 were proposed to improve the robustness of LDA-L2 and 2DLDA-L2 [13]. The high time complexity will be encountered for LDA-R1 when the dimensionality of the training data is high. Similarly, inspired by PCA-L1 [10] and CSP-L1 [14], LDA-L1 [15], [16] was proposed to further improve the robustness of LDA-L2 and the performance of LDA-R1. Subsequently, Li et al. [17] extended LDA-L1 to two-dimensional linear discriminant analysis (L1-2DLDA) for making full use of the local structure of images. But the optimization of L1-2DLDA cannot obtain the solution of the corresponding trace ratio form [18]. Recently, Liu et al. [19] proposed an iterative non-greedy framework to optimize the trace ratio form of LDA-L1 and analyzed the convergence of their proposed algorithm. The idea of LDA-L1 was extended to optimize the objective function of 2DLDA-L1 [18].

Correntropy was proposed as a similarity metric which reflects the second-order statistics of the input data in the transformed space [20]. Theoretically, maximum correntropy criterion (MCC) is a local similarity criterion of data pairs and should have outstanding advantages when large nonzero mean and non-Gaussian outliers are present [20]. In 2011, He et al. [21] proposed a robust PCA based on MCC (PCA-MCC) by applying correntropy to measure the construction error. PCA-MCC outperforms PCA-R1 and PCA-L1 when there are large outliers in the training data. He et al. [22] further proposed a robust face recognition method to validate the feasibility of correntropy. Subsequently, MCC was used to improve LDA [23] and LPP [24] against outliers respectively. Recently, correntropy was successfully applied to multidimensional scaling [25], tensor factorization [26] and multi-label active learning [27]. The aforementioned methods adequately demonstrated the robustness of correntropy.

Although 2DLDA-R1 [13] and 2DLDA-L1 [18] are capable of alleviating the impact of outliers at a certain level, they are difficult to cope with large nonzero mean and non-Gaussian outliers. Correntropy has been theoretically proved to be a robust metric for dealing with large nonzero mean and non-Gaussian outliers [20]. In practice, several applications also have illustrated that MCC is very robust against outliers [21–27]. In this paper, therefore, correntropy will be applied to further enhance the robustness of 2DLDA methods against outliers and a new robust 2DLDA method based on MCC (2DLDA-MCC) is proposed. 2DLDA-MCC adopts the within-class similarity based on correntropy to alleviate the negative effect of outliers, and aims to obtain the optimal projection transformation by maximizing the correntropy-based within-class similarity and simultaneously maintaining the global dispersity. Moreover, the objective problem of 2DLDA-MCC is solved by an iterative half-quadratic optimization procedure. Compared with the existing 2DLDA methods, 2DLDA-MCC has the following two appealing aspects.

• Traditional 2DLDA methods often apply the same metric (i.e. Frobenius-norm, R1-norm or L1-norm) to measure the within-class similarity and the global dispersity in their objective problems. 2DLDA-MCC adopts two different metrics to measure the within-class similarity and the global dispersity respectively. The main advantage is that LDA-MCC simultaneously maintains the within-class cohesion owing the robustness of correntropy against outliers and the global dispersity owing the property of Frobenius-norm. The idea may inspire the potential researches on robust supervised subspace learning with multiple metrics.

• Actually, the proposed optimization procedure of 2DLDA-MCC is a reweighted method, and the Gaussian like weighting function attenuates the large outlying terms so that outliers would have a less impact on the adaptation during the optimization procedure [20]. Therefore, 2DLDA-MCC shows better robustness on overcoming outliers compared with the traditional 2DLDA methods and several experiments illustrate the results.

The remainder of this paper is organized as follows. In Section 2, the objective problem of 2DLDA-MCC is formulated and its optimization algorithm is given in detail. In Section 3, we report the experimental results on three image databases. Finally, Section 4 simply concludes this paper.