Two-dimensional canonical correlation analysis and its application in small sample size face recognition

Abstract

In the traditional canonical correlation analysis (CCA)-based face recognition methods, the size of sample is always smaller than the dimension of sample. This problem is so-called the small sample size (SSS) problem. In order to solve this problem, a new supervised learning method called two-dimensional CCA (2DCCA) is developed in this paper. Different from traditional CCA method, 2DCCA directly extracts the features from image matrix rather than matrix to vector transformation. In practice, the covariance matrix extracted by 2DCCA is always full rank. Hence, the SSS problem can be effectively dealt with by this new developed method. The theory foundation of 2DCCA method is first developed, and the construction method for the class-membership matrix Y which is used to precisely represent the relationship between samples and classes in the 2DCCA framework is then clarified. Simultaneously, the analytic form of the generalized inverse of such class-membership matrix is derived. From our experiment results on face recognition, we clearly find that not only the SSS problem can be effectively solved, but also better recognition performance than several other CCA-based methods has been achieved.

Appendix: the generalized inverse of class-membership covariance matrix

From (5), we can obtain:

$$ \begin{aligned} \Upsigma_{YY} & = \sum\limits_{i = 1}^{C} {\sum\limits_{j = 1}^{{n_{i} }} {(y_{ij} - \overline{Y} )} } (y_{ij} - \overline{Y} )^{\text{T}} \\ & = Y\left(I_{l \times N} - \frac{1}{N}R_{N} R_{N}^{\text{T}} \right)Y^{\text{T}} = YY^{\text{T}} - \frac{1}{N}YR_{N} R_{N}^{\text{T}} Y^{\text{T}} = U - \frac{1}{N}UR_{C} R_{C}^{\text{T}} U^{\text{T}} x \\ \end{aligned} $$

(15)

$$ \begin{aligned} \Upsigma_{ZZ} & = \sum\limits_{i = 1}^{C} {\sum\limits_{j = 1}^{{n_{i} }} {(z_{ij} - \overline{Z} )} } (z_{ij} - \overline{Z} )^{\text{T}} \\ & = Z\left(I_{l \times N} - \frac{1}{N}R_{N} R_{N}^{\text{T}} \right)Z^{\text{T}} = ZZ^{\text{T}} - \frac{1}{N}ZR_{N} R_{N}^{\text{T}} Z^{\text{T}} = M - \frac{1}{N}MR_{C - 1} R_{C - 1}^{\text{T}} M \\ \end{aligned} $$

(16)

where $\left( {I_{l \times N} - \frac{1}{N}R_{N} R_{N}^{\text{T}} } \right)$ is centering projector, I _l×N is the (l × N) × (l × N) identity matrix, and R ^T _N is a matrix, which size is h × (l × N), composed of Nnumber of Q matrix

$$ \left( {I_{l \times N} - \frac{1}{N}R_{N} R_{N}^{\text{T}} } \right) = \left( {\left[ {\begin{array}{*{20}c} 1 & 0 & \cdots & 0 \\ 0 & 1 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & 1 \\ \end{array} } \right] - \frac{1}{N}\left[ {\begin{array}{*{20}c} {Q^{\text{T}} } \\ {Q^{\text{T}} } \\ \vdots \\ {Q^{T} } \\ \end{array} } \right]\left[ {\begin{array}{*{20}c} Q & Q & \cdots & Q \\ \end{array} } \right]} \right) $$

(17)

and

$$ M = \left[ {\begin{array}{*{20}c} {S_{1} } & 0 & 0 & 0 \\ 0 & {S_{2} } & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & 0 & \cdots & {S_{C - 1} } \\ \end{array} } \right]_{{\{ h \times (C - 1)\} \times \{ h \times (C - 1)\} }} $$

(18)

$$ U = \left[ {\begin{array}{*{20}c} M & 0 \\ 0 & {S_{C} } \\ \end{array} } \right]_{(h \times C) \times (h \times C)} $$

(19)

where $S_{i} = n_{i} QQ^{\text{T}} .$ Because M is diagonal matrix, the generalized inverse M ^G is also diagonal matrix and the non-zero elements in the diagonal is equal to the reciprocal of the corresponding elements in diagonal of M. From literature [17, 18], the generalized inverse of Σ _ZZ is $\Upsigma_{ZZ}^{G} = (({1 \mathord{\left/ {\vphantom {1 {n_{C} }}} \right. \kern-\nulldelimiterspace} {n_{C} }})R_{C - 1} R_{C - 1}^{\text{T}} + M^{G} ).$

From (19), the matrix Σ _YY can be written as

$$ \Upsigma_{YY} = \left[ {\begin{array}{*{20}c} {\Upsigma_{ZZ} } & V \\ {V^{T} } & {\frac{{N - n_{C} }}{N}S_{C} } \\ \end{array} } \right]_{(h \times C) \times (h \times C)} $$

(20)

where

$$ V = \frac{{ - n_{C} }}{N}MR_{C - 1} $$

(21)

So the generalized inverse of class-membership covariance matrix is as following

$$ \Upsigma_{YY}^{g} = \left[ {\begin{array}{*{20}l} {\Upsigma_{ZZ}^{g} } & {0_{{\{ h \times (C - 1)\} h}} } \\ {0_{{h \times \{ h \times (C - 1)\} }} } & {0_{h \times h} } \\ \end{array} } \right]_{(h \times C) \times (h \times C)} $$

(22)