Hallucinating Compressed Face Images

Abstract

A face hallucination algorithm is proposed to generate high-resolution images from JPEG compressed low-resolution inputs by decomposing a deblocked face image into structural regions such as facial components and non-structural regions like the background. For structural regions, landmarks are used to retrieve adequate high-resolution component exemplars in a large dataset based on the estimated head pose and illumination condition. For non-structural regions, an efficient generic super resolution algorithm is applied to generate high-resolution counterparts. Two sets of gradient maps extracted from these two regions are combined to guide an optimization process of generating the hallucination image. Numerous experimental results demonstrate that the proposed algorithm performs favorably against the state-of-the-art hallucination methods on JPEG compressed face images with different poses, expressions, and illumination conditions.

Appendix

Solving (2) Given a LR image D(L), and gradient maps U, we generate a HR image $H^*$ through

$$\begin{aligned} H^* = \underset{H}{{\text {argmin}}} \Vert \nabla H - U\Vert ^2\;\; \text{ s.t. } \;\;(H \otimes G_\sigma ) \downarrow _s = D(L). \end{aligned}$$

To handle the nonlinear constraint, we relax the problem by

$$\begin{aligned} H^* = \underset{H}{{\text {argmin}}} \; \Vert \nabla H - U\Vert ^2 + \beta \Vert (H\otimes G_\sigma )\downarrow _s - D(L)\Vert ^2, \nonumber \\ \end{aligned}$$

(9)

where $\beta$ is a weight parameter. We use the gradient descent method to solve the optimization problem.

Algorithm 1 shows the details how (9) is solved. The original energy value e is computed on Line 9, and a descent direction for generating a new image is computed on Line 10, where the $\text{ Div }(\cdot )$ is a divergence operator and $U^k$ means the k-th map in U for one of the eight derivative directions. We carry out a line search on Lines 11 to 15 and record the energy values of all step lengths in an array r. We find the best step index $j^*$ and check the energy value $r[j^*]$ on Line 17. If the new energy value $r[j^*]$ is smaller than the original energy value e, the image is updated on Lines 18 to 19.