Abstract
Face Super-Resolution (FSR) is to infer high resolution facial image(s) from given low resolution one(s). But when large-scale training samples are absent, FSR may fail in inferring high resolution image for practical low resolution facial image with complex degradation. To solve this problem, we present a novel position patch-based FSR method via latent Deformable Constrained Model (FSR-DCM). Different from conventional FSR methods that view an image patch as a fixed-length vector, we train the target image patch as a matrix in a flexible deformation flow form. This enables the dictionary to cover patterns that do not appear in training examples, resulting in our FSR method to be more expressive and able to solve the outlier (heterogeneous) problem. Besides, instead of explicit Euclidean distance, we use latent deformable similarity as distance criteria to measure the patch similarity, which facilitates our FSR method to work in low-quality scene by emphasizing neighbor relationship and enlarging the distance difference. Through enforcing such a constraint, the expressive capability of the FSR method can be improved, and the restoration failure caused by the coefficients of wrongly emphasized candidates can be overcome. Experiments on the public face datasets CAS-PEAL-R1 [9] and FEI [25] demonstrate the superiority of the proposed algorithm against the existing solutions to the problem of enhancing facial images of very low resolution both quantitatively (Peak-Singal-to-Noise Ratio, i.e., PSNR and Structural Similarity, i.e., the SSIM [29]) and qualitatively (subjective performance).
Similar content being viewed by others
References
Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183
Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel PCA-based prior. IEEE Trans Multimed 9(4):888–892
Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. Comput Vis Pattern Recogn 1:275–282
Chen L, Hu R, Jiang J, Han Z (2014) Efficient learning based face hallucination approach via facial standard deviation prior. In: IEEE International Symposium on Circuits and Systems, pp 2057–2060
Chen L, Hu R, Liang C, Li Q, Han Z (2017) A novel face super resolution approach for noisy images using contour feature and standard deviation prior. Multimed Tool Appl 76(2):2467–2493
Dong X, Hu R, Jiang J, Han Z, Chen L, Gao G (2014) Noise face image hallucination via data-driven local eigentransformation. In: Pacific Rim Conference on Multimedia, pp 183–192
Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307
Everingham M, Sivic J, Zisserman A (2006) Hello! My name is... Buffy–automatic naming of characters in TV video. In: British Machine Vision Conference, pp 899–908
Gao W, Cao B, Shan S (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations[J]
Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. International Conference on Computer Vision 349–356
Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications. IEEE Trans Circ Syst Video Technol 21(10):1363–1377
Jiang J, Hu R, Han Z, Huang K, Lu T (2013) Graph discriminant analysis on multi-manifold (GDAMM): a novel super-resolution method for face recognition. IEEE International Conference on Image Processing 1465–1468
Jiang J, Hu R, Liang C, Han Z, Zhang C (2014) Face image super-resolution through locality-induced support regression. Signal Process 103:168–183
Jiang J, Hu R, Wang Z, Han Z (2014) Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans Image Process 23(10):4220–4231
Jiang J, Hu R, Wang Z, Han Z (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281
Lan C, Hu R, Huang K, Han Z (2010) Face hallucination with shape parameters projection constraint. In: Proceedings of the ACM international conference on Multimedia, pp 883–886
Lee H, Battle A, Raina R, Ng AY (2007) Efficient sparse coding algorithms. In: Advances in neural information processing systems, pp 801–808
Liu S, Yang MH (2014) Compressed face hallucination. In: IEEE International Conference on Image Processing, pp 4032–4036
Liu C, Yuen J, Torralba A (2011) Sift flow: dense correspondence across scenes and its applications. IEEE Trans Pattern Anal Mach Intell 33(5):978–994
Lucas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp 674–679
Ma X, Zhang J, Qi C (2009) Position-based face hallucination method. In: ICME 2009 IEEE International Conference on Multimedia and Expo
Qu S, Hu R, Chen S, Jiang J, Wang Z, Chen J (2014) Face hallucination via re-identified K-nearest neighbors embedding. In: IEEE International Conference on Multimedia and Expo, pp 1–6
Rowley H, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38
Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. Comput Vis Pattern Recogn 2, II:729–36
Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913
Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Royal Stat Soc Series B (Methodological) 267–288
Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision, pp 1920–1927
Wang X, Tang X (2005) Hallucinating face by eigentransformation . IEEE Trans Syst, Man, Cybern Part C (Applications and Reviews) 35(3):425–434
Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612
Yang J, Tang H, Ma Y, Huang T (2008) Face hallucination via sparse coding. In: IEEE International Conference on Image Processing, pp 1264–1267
Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873
Yang CY, Liu S, Yang MH (2013) Structured face hallucination. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1099–1106
Zhou E, Fan H, Cao Z, Jiang Y, Yin Q (2015) Learning face hallucination in the wild. Assoc Advan Artif Intell 3871–3877
Acknowledgements
This work was supported in part by the National Nature Science Foundation of China under Grant No. 61901117, U1805262, in part by the Natural Science Foundation of Fujian Province under the Grant 2019J05060 and Grant 2019J01271, in part by the Fujian Provincial Education Department Project under Grant JT180094 and Grant JT180095, in part by the Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, under Grant HBIR201906, in part by the Special Funds of the Central Government Guiding Local Science and Technology Development under Grant 2017L3009, and in part by the National Key Research and Development Program of China under Grant 2016YFB1001001.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Chen, L., Li, Q. & Jiang, J. Noisy practical facial super-resolution method via deformable constrained model with small dataset. Multimed Tools Appl 79, 2577–2600 (2020). https://doi.org/10.1007/s11042-019-08277-7
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-019-08277-7