Abstract
In this paper we propose a novel face hallucination algorithm to synthesize a high-resolution face image from several low-resolution input face images. Face hallucination normally uses two models: a global parametric model which synthesizes the global face shapes from eigenfaces, and a local parametric model which enhances the local high frequency details. We follow a similar process to develop a robust face hallucination algorithm. First, we obtain eigenfaces from a number of low resolution face images segmented from a video sequence using a face tracking algorithm. Then we compute the difference between an interpolated low-resolution face and a mean face, and use this difference as the query to retrieve an approximate sparse eigenface representation. The eigenfaces are combined using the coefficients obtained from the sparse representation and added into the interpolated low-resolution face. In this way, the global shape of the interpolated low resolution face can be successfully enhanced. Second, we improve the example-based super-resolution method for local high frequency information enhancement. Our proposed algorithm uses the Approximate Nearest Neighbors (ANN) search method to find a number of nearest neighbors for a stack of queries, instead of finding the exact match for each low frequency patch. Median filtering is used to remove the noise from the nearest neighbors in order to enhance the signal. Our proposed algorithm uses a sparse representation and the ANN method to enhance both global face shape and local high frequency information while greatly improving the processing speed, as confirmed empirically.
Similar content being viewed by others
References
Baker S, Kanade T (2000) Hallucinating faces. In: Proceedings of IEEE international conference on automatic face and gesture recognition
Blackman S (2004) Multiple hypothesis tracking for multiple target tracking. IEEE Aerosp Electron Syst Mag 19(1):5–18
Elad M, Aharon M (2006) Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans Image Process 15:3736–374
Farsiu S, Robinson MD, Elad M, Milanfar P (2004) Fast and robust multiframe super resolution. IEEE Trans Image Process 13(10):1327–1344
Freeman WT, Jones TR, Pasztor EC (2002) Example-based super-resolution. IEEE Comput Graph Appl 22(2):56–65
Hu Y, Lam, K-M, Qiu G, Shen T (2011) From local pixel structure to global image super-resolution: a new face hallucination framework. IEEE Trans Image Process 20:433–445
Jian M, Lam K-M, Dong J (2013) A novel face-hallucination scheme based on singular value decomposition. Patt Recogn 46(11):3091–3102
Jung C, Jiao L, Liu B, Gong M (2011) Position-patch based face hallucination using convex optimization. IEEE Signal Process Lett 18:367–370
Liu C, Shum HY, Freeman WT (2007) Face hallucination: theory and practice. Int J Comput Vis 75(1):115–134
Liu T, Moore AW, Yang K, Gray AG (2004) An investigation of practical approximate nearest neighbor algorithms. In: Advances in neural information processing systems, pp 825–832
Ma X, Huang H, Wang S, Qi C (2010) A simple approach to multiview face hallucination. IEEE Signal Process Lett 17:579–582
Turk M, Pentland A (1991) Face recognition using eigenfaces. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 586–591
Viola P, Jones MJ (2004) Robust real-time face detection. Int J Comput Vis 57(2):137–154
Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227
Yang J, Wright J, Huang T, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19:2861–2873
Yang M-H, Kriegman D, Ahuja N (2002) Detecting faces in images: a survey. IEEE Trans Pattern Anal Mach Intell 24(1):34–58
Yilmaz A, Javed O, Shah M (2006) Object tracking: a survey. ACM Comput Surv. doi:10.1145/1177352.1177355
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Jia, Z., Zhao, J., Wang, H. et al. A two-step face hallucination approach for video surveillance applications. Multimed Tools Appl 74, 1845–1862 (2015). https://doi.org/10.1007/s11042-013-1721-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-013-1721-4