Abstract
Face Super Resolution (FSR) is to infer high resolution (HR) face images from given low resolution (LR) face images with the help of HR/LR training examples. The most representative FSR is NE methods, which are based on the consistency assumption that the HR/LR patch pairs form similar local geometric structures. But NE methods have difficulty in dealing with noisy facial images. The reason lies in the wrong neighborhood relationship caused by low quality scenarios that even two distinct patches have similar relation in local geometry. Therefore, the consistency assumption is not well held anymore. This paper presents a novel FSR approach suitable for noisy facial images. Our work are twofold. Firstly, different from the existing methods which directly enhance the noisy input image in intensity feature space, the proposed method introduces a contour feature which is robust to noise. By applying the contour feature as constraint, the noise effects can be effectively suppressed. Secondly, different from the existing methods which directly constrain the noisy input image with low quality contour feature, a standard deviation prior is proposed to enhance the low quality contour feature. Through enhancing the contour feature into high quality, the FSR reconstruction can be better constrained. Both simulation and the real-world scenario experiments demonstrate that the proposed approach outperforms most classic methods both quantitatively and qualitatively.
Similar content being viewed by others
References
Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183
Bowyer KW, Chang K, Flynn P (2006) A survey of approaches and challenges in 3D and multi-modal 3D+ 2D face recognition. Computer vision and image understanding 101.1, pp 1–15
Baker S, Kanade T (2000) Hallucinating faces[C]. In: Proceedings of the 4th IEEE international conference on automatic face and gesture recognition, 2000. IEEE, pp 83–88
Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183
Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising[C]. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 2. IEEE, pp 60–65
Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding[C]. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. IEEE, p 1: I-I
Chen L, Hu R, Han Z et al (2013) A joint learning based face hallucination approach for low quality face image[C]. In: 20th IEEE international conference on image processing (ICIP), 2013. IEEE, p 972
Chen L, Hu R, Jiang J et al (2014) Efficient learning based face hallucination approach via facial standard deviation prior[C]. In: IEEE international symposium on circuits and systems (ISCAS), 2014. IEEE, pp 2057–2060
Chang KI, Bowyer KW, Flynn PJ (2005) An evaluation of multimodal 2D+ 3D face biometrics. In: IEEE Transactions on 27.4 Pattern Analysis and Machine Intelligence, pp 619–624
Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel PCA-based prior. IEEE Trans Multimed 9(4):888–892
Chan TM, Zhang J, Pu J et al (2009) Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recogn Lett 30(5):494–502
Dong X, Hu R, Jiang J et al (2014) Noise face image hallucination via data-driven local eigentransformation[M]. In: Advances in multimedia information processing C PCM 2014. Springer International Publishing, pp 183–192
Gao W, Cao B, Shan S et al (2008) The CAS-PEAL large-scale chinese face database and baseline evaluations. Humans IEEE Trans Syst Man, Cybern Part A Syst 38(1):149–161
Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications. IEEE Trans Circ Syst Video Technol 21(10):1363–1377
Jiang J, Hu R, Liang C et al (2014) Face image super-resolution through locality-induced support regression. Signal Process 103(10):168–183
Jiang J, Hu R, Wang Z et al (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281
Jiang J, Hu R, Liang C, Han Z, Zhang C (2014) Face image super-resolution through locality-induced support regression. Signal Process 103:168–183
Jiang J, Hu R, Wang Z et al (2015) CDMMA: coupled discriminant multi-manifold analysis for matching low-resolution face images[J]. Signal Process
Jiang J, Ma X, Cai Z et al (2015) Sparse support regression for image super-esolution[J]. Photonics J IEEE 7(5):1–11
Jiang J, Hu R, Wang Z, Han Z, Ma J (2015) Facial image hallucination through coupled-layer neighbor embedding. IEEE Transactions on Circuits and Systems for Video Technology. doi:10.1109/TCSVT.2015.2433538
Kolouri S, Rohde GK (2015) Transport-based single frame super resolution of very low resolution face images[C]. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 4876–4884
Lin Z, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans Pattern Anal Mach Intell 26(1):83–97
Liu C, Shum HY, Zhang CS (2001) A two-step approach to hallucinating faces: global parametric model and local nonparametric model[C]. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, CVPR 2001, vol 1. IEEE, pp 1: I-192-I-198
Li B, Chang H, Shan S et al (2009) Aligning coupled manifolds for face hallucination. IEEE Signal Process Lett 16(11):957–960
Lan C, Hu R, Huang K et al (2010) Face hallucination with shape parameters projection constraint[C]. In: Proceedings of the international conference on multimedia. ACM, pp 883–886
Liu S, Yang M H (2014) Compressed face hallucination[C]. In: IEEE international conference on image processing (ICIP), 2014. IEEE, pp 4032–4036
Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527
Ma X, Zhang J, Qi C (2009) Position-based face hallucination method[C]. In: IEEE international conference on multimedia and expo, ICME, 2009. IEEE, pp 290–293
Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36
Park SW, Savvides M (2007) Breaking the limitation of manifold analysis for super-resolution of facial images[C]. Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE Int Conf I-573-I-576
Rowley H, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38
Shao L, Yan R, Li X et al (2014) From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms. IEEE Trans Cybern 44(7):1001–1013
Shao L, Zhang H, De HG (2008) An overview and performance evaluation of classification-based least squares trained filters. IEEE Trans Image Process 17 (10):1772–1782
Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, vol 2. IEEE, pp II-729
Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650
Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913
Vezzetti E, Calignano F, Moos S (2010) Computer-aided morphological analysis for maxillo-facial diagnostic: a preliminary study. Journal of Plastic, Reconstructive, Aesthetic Surgery 63.2, pp 218–226
Vezzetti E, Marcolin F, Stola V (2013) 3D human face soft tissues landmarking method: An advanced approach, Computers in Industry 64.9, pp 1326–1354
Wang X, Tang X (2005) Hallucinating face by eigentransformation. IEEE Trans Syst Man Cybern Part C: Appl Rev 35(3):425–434
Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612
Yan R, Shao L, Liu Y (2013) Nonlocal hierarchical dictionary learning using wavelets for image denoising. IEEE Trans Image Process 22(12):4689–4698
Yang CY, Liu S, Yang MH (2013) Structured face hallucination[C]. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1099–1106
Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873
Zou WWW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340. doi:10.1109/TIP.2011.2162432
Acknowledgments
The research is supported by the National Nature Science Foundation of China (No. 61231015, 61172173, 61201247, U1404618, 61303114), the National High Technology Research and Development Program of China (863 Program), Technology Research Program of Ministry of Public Security (No.2014JSYJA016), Guangdong-Hongkong Key Domain Breakthrough Project of China (No. 2012A090200007), The major Science and Technology Innovation Plan of Hubei Province(No. 2013AAA020), China Postdoctoral Science Foundation funded project (2013M530350), Specialized Research Fund for the Doctoral Program of Higher Education (No. 20130141120024), Key Technology RD Program of Wuhan(2013030409020109).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Chen, L., Hu, R., Liang, C. et al. A novel face super resolution approach for noisy images using contour feature and standard deviation prior. Multimed Tools Appl 76, 2467–2493 (2017). https://doi.org/10.1007/s11042-015-3145-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-015-3145-9