Abstract
Face Super Resolution (FSR) is to infer High Resolution (HR) facial images from given Low Resolution (LR) ones with the assistance of LR and HR training pairs. Among existing methods, Neighbor Embedding(NE) FSR methods are superior in visual and objective quality than holistic based methods. These NE methods are based on the consistency assumption that the neighbors in HR/LR space form similar local geometry. But when LR images are in Very Low Quality (VLQ), the LR patches are seriously contaminated that even two distinct patches form similar appearance, which means that the consistency assumption is not well held anymore. To solve this problem, in this paper we use the target patch as well as the surrounding pixels, which we call parent patch, to represent the target patch. By incorporating the peripheral information, the parent patch is much more robust to noise in the LR and HR consistency learning. The effectiveness of proposed method is verified both quantitatively and qualitatively. In this paper, we also discuss the boundary and the paradox of the multi-scaled parent patch prior in NE based FSR framework.
Similar content being viewed by others
References
Baker S, Kanade T (2002) Limits on super-resolution and how to break them[J]. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183
Baker S, Kanade T (2000) Hallucinating faces[C]. In: Proceedings. Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000. IEEE, pp 83–88
Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding[C]. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. IEEE, pp 1: I-I
Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel PCA-based prior[J]. IEEE Trans Multimed 9(4):888–892
Dong X, Hu R, Jiang J et al (2014) Noise Face Image Hallucination via Data-Driven Local Eigentransformation[M]. In: Advances in Multimedia Information Processing C PCM 2014Springer International Publishing, pp 183–192
Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image[C]. In: 2009 IEEE 12th International Conference on Computer Vision. IEEE, pp 349–356
Gao W, Cao B, Shan S et al (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations[J]. IEEE Tran Syst Man Cybern Part A Syst Human 38(1):149–161
Huang H, Wu N (2011) Fast Facial Image Super-Resolution via Local Linear Transformations for Resource-Limited Applications[J]. IEEE Trans Circ Syst Video Technol 21(10):1363–1377
Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications[J]. IEEE Trans Circ Syst Video Technol 21(10):1363–1377
Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications[J]. IEEE Trans Circ Syst Video Technol 21(10):1363–1377
Jiang J, Hu R, Liang C et al (2014) Face image super-resolution through locality-induced support regression[J]. Signal Process 103(10):168–183
Jiang J, Hu R, Wang Z et al (2014) Noise robust face hallucination via locality-constrained representation[J]. IEEE Trans Multimed 16(5):1268–1281
Jiang J, Hu R, Wang Z et al (2015) CDMMA: Coupled discriminant multi-manifold analysis for matching low-resolution face images[J]
Jiang J, Ma X, Cai Z et al (2015) Sparse Support Regression for Image Super-Resolution[J]. IEEE Photon J 7(5):1–11
Jiang J, Hu R, Liang C, Han Z, Zhang C (2014) Face image super-resolution through locality-induced support regression. Signal Process 103:168–183
Jiang J, Hu R, Wang Z, Han Z, Ma J Facial Image Hallucination through Coupled-Layer Neighbor Embedding
Kolouri S, Rohde GK (2015) Transport-based single frame super resolution of very low resolution face images[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4876–4884
Lin Z, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation[J]. IEEE Trans Pattern Anal Mach Intell 26(1):83–97
Liu C, Shum HY, Zhang CS (2001) A two-step approach to hallucinating faces: global parametric model and local nonparametric model[C]. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol 1. IEEE, pp 1: I-192-I-198
Li B, Chang H, Shan S et al (2009) Aligning coupled manifolds for face hallucination[J]. IEEE Signal Process Lett 16(11):957–960
Lan C, Hu R, Huang K et al (2010) Face hallucination with shape parameters projection constraint[C]. In: Proceedings of the international conference on Multimedia. ACM, pp 883–886
Liu S, Yang MH (2014) Compressed face hallucination[C]. In: 2014 IEEE International Conference on Image Processing (ICIP). IEEE, pp 4032–4036
Ma X, Zhang J, Qi C (2009) Position-based face hallucination method[C]. In: IEEE International Conference on Multimedia and Expo, 2009. ICME 2009. IEEE, pp 290–293
Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview[J]. IEEE Signal Process Mag 20(3):21–36
Peng C, Gao X, Wang N et al (2015) Multiple Representations-Based Face Sketch-Photo Synthesis[J]
Shao L, Yan R, Li X et al (2014) From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms[J]. IEEE Trans Cybern 44(7):1001–1013
Shao L, Zhang H, De Haan G (2008) An overview and performance evaluation of classification-based least squares trained filters[J]. IEEE Trans Image Process 17 (10):1772–1782
Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. In: Proceedings. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, vol 2. IEEE, pp II-729
Su K, Tian Q, Xue Q et al (2005) Neighborhood issue in single-frame image super-resolution[C]. In: IEEE International Conference on Multimedia and Expo, 2005. ICME 2005. IEEE, p 4
Wang N, Tao D, Gao X et al (2014) A comprehensive survey to face hallucination[J]. Int J Comput Vis 106(1):9–30
Wang X, Tang X (2005) Hallucinating face by eigentransformation[J]. IEEE Trans Syst Man Cybern Part C Appl Rev 35(3):425–434
Wang N, Li J, Tao D, Li X, Gao X (2013) Heterogeneous image transformation. Pattern Recogn Lett 34(1):77–84
Wang N, Tao D, Gao X, Li X, Li J (2013) Transductive face sketch-photo synthesis. IEEE Trans Neural Netw Learn Syst 24(9):1364–1376
Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity[J]. IEEE Trans Image Process 13(4):600–612
Wang Z, Hu R, Wang S et al (2014) Face hallucination via weighted adaptive sparse regularization[J]. IEEE Trans Circ Syst Video Technol 24(5):802–813
Yan R, Shao L, Liu Y (2013) Nonlocal hierarchical dictionary learning using wavelets for image denoising[J]. IEEE Trans Image Process 22(12):4689–4698
Yang CY, Liu S, Yang MH (2013) Structured face hallucination[C]. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 1099–1106
Yang J, Tang H, Ma Y et al (2008) Face hallucination via sparse coding[C]. In: 15th IEEE International Conference on Image Processing, 2008. ICIP 2008. IEEE, pp 1264–1267
Zou WWW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327,340. doi:10.1109/TIP.2011.2162432
Acknowledgments
The research is supported by the National High Technology Research and Development Program of China (863 Program No. 2015AA016306); the National Nature Science Foundation of China (No. 61231015, 61172173, 61303114, U1404618, 61501413, 61502354); the Internet of Things Development Funding Project of Ministry of industry in 2013(No. 25); the Technology Research Program of Ministry of Public Security (No. 2014JSYJA016); the China Postdoctoral Science Foundation funded project (2013M530350, 2014M562058); the Specialized Research Fund for the Doctoral Program of Higher Education (No. 20130141120024); the Fundamental Research Funds for the Central Universities(2042014kf0025). This work was partly supported by the EU FP7 QUICK project under Grant Agreement No. PIRSES-GA-2013-612652. The authors would like to thank Dr. Junjun Jiang for his important contribution in this paper.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chen, L., Hu, R., Han, Z. et al. Face super resolution based on parent patch prior for VLQ scenarios. Multimed Tools Appl 76, 10231–10254 (2017). https://doi.org/10.1007/s11042-016-3611-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-016-3611-z