A robust face super-resolution algorithm and its application in low-resolution face recognition system

Abstract

In real-world surveillance scenario, the face recognition (FR) systems pose a lot of challenges due to the captured low-resolution (LR) and noisy probe images. A new face super-resolution (SR) algorithm is proposed to design a recognition model overcoming the challenges of existing FR systems. The proposed SR algorithm inherits the merits of functional-interpolation and dictionary-based SR techniques. The functional interpolation assists in generating more discriminable output, whereas the dictionary-based approach assists in eliminating noise effects from the reconstruction process. Consequently, it produces more discriminable and noise-free high-resolution (HR) images from captured noisy LR probe images, suitable for real-world problems like low-resolution face recognition. The results obtained from the experiments performed on several popular face image datasets including FEI, FERET, and CAS-PEAL-R1 show that the proposed algorithm performs better than all the comparative SR methods.

Appendices

Appendix A

The matrix form of Eq. (1) is given below:

$$ \begin{aligned} w^{\ast}(a,b) & = \left\Vert {\varTheta}(a,b) \odot \left( P^{L}(a,b) - G^{L}(a,b)w(a,b) \right)\right\Vert_{2}^{2} \cr & \qquad + \tau \Vert Sw(a,b){\Vert_{2}^{2}} \end{aligned} $$

(12)

where G^L (a,b) is the matrix in which each column correspond to one LR dictionary patch. After multiply Θ(a, b) with each element we obtain,

$$ \begin{aligned} w^{\ast}(a,b) = \left\Vert \ddot{P^{L}}(a,b) - \ddot{G^{L}}(a,b) w(a,b) \right\Vert_{2}^{2} \quad + \tau \Vert Sw(a,b){\Vert_{2}^{2}} \end{aligned} $$

(13)

where $\ddot {P^{L}}(a,b) = {\varTheta }(a,b) \odot P^{L}(a,b)$ and $\ddot {G^{L}}(a,b) = {\varTheta }(a,b) \odot G^{L}(a,b)$.

Following [4, 32, 40], (13) can be simplified as:

$$ \begin{aligned} w(a,b) = \left( C(a,b) + \tau S^{2}\right) \setminus ones(N,1) \end{aligned} $$

(14)

where term ones(N, 1) denotes column vector of value one (or 1), operator “∖” is the left matrix devision, and C(a, b) denotes covariance matrix for P^L(a, b) which is calculated as:

$$ \begin{aligned} C(a,b) = J^{T}J, \end{aligned} $$

(15)

where J = P^L(a, b)ones(N, 1)^T −G^L(a, b). Finally, obtained weights are rescaled to satisfy $\sum \limits _{n = 1}^{N} {{w_n}(a,b)} = 1$.

Appendix B

Since the objective function given in (9) is convex w.r.t.Λ, the optimal Λ^∗ is obtain by taking its derivative w.r.t. Λ as follows:

$$ \begin{aligned} \frac{\partial {\varLambda}^{\ast}}{\partial {\varLambda}} = \frac{\partial \left( G_{mp}^{H} - {\varLambda} G^{L}\right) {\varGamma} \left( G_{mp}^{H} - {\varLambda} G^{L}\right)^{T} }{\partial {\varLambda}} = 0, \end{aligned} $$

(16)

$$ \begin{aligned} -2G_{mp}^{H}{\varGamma} {G^{L}}^{T} + 2G^{L}{\varGamma} {G^{L}}^{T} {\varLambda} = 0, \end{aligned} $$

(17)

$$ \begin{aligned} {\varLambda} = \frac{G_{mp}^{H}{\varGamma} {G^{L}}^{T}}{G^{L}{\varGamma} {G^{L}}^{T}}, \end{aligned} $$

(18)

To make term G^LΓG^LT invertible (non-singular), by following [10], we add a small constant value λ to the diagonal elements of the matrix G^LΓG^LT before taking its inverse. Thus we have,

$$ \begin{aligned} {\varLambda} = G_{mp}^{H}{\varGamma} {G^{L}}^{T} \left( G^{L}{\varGamma} {G^{L}}^{T} + \lambda I \right)^{-1} \end{aligned} $$

(19)

For our experiments, the λ is set to small constant value 10^− 6.