Skip to main content
SpringerLink
Log in

Noisy practical facial super-resolution method via deformable constrained model with small dataset

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face Super-Resolution (FSR) is to infer high resolution facial image(s) from given low resolution one(s). But when large-scale training samples are absent, FSR may fail in inferring high resolution image for practical low resolution facial image with complex degradation. To solve this problem, we present a novel position patch-based FSR method via latent Deformable Constrained Model (FSR-DCM). Different from conventional FSR methods that view an image patch as a fixed-length vector, we train the target image patch as a matrix in a flexible deformation flow form. This enables the dictionary to cover patterns that do not appear in training examples, resulting in our FSR method to be more expressive and able to solve the outlier (heterogeneous) problem. Besides, instead of explicit Euclidean distance, we use latent deformable similarity as distance criteria to measure the patch similarity, which facilitates our FSR method to work in low-quality scene by emphasizing neighbor relationship and enlarging the distance difference. Through enforcing such a constraint, the expressive capability of the FSR method can be improved, and the restoration failure caused by the coefficients of wrongly emphasized candidates can be overcome. Experiments on the public face datasets CAS-PEAL-R1 [9] and FEI [25] demonstrate the superiority of the proposed algorithm against the existing solutions to the problem of enhancing facial images of very low resolution both quantitatively (Peak-Singal-to-Noise Ratio, i.e., PSNR and Structural Similarity, i.e., the SSIM [29]) and qualitatively (subjective performance).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183

    Article  Google Scholar 

  2. Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel PCA-based prior. IEEE Trans Multimed 9(4):888–892

    Article  Google Scholar 

  3. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding. Comput Vis Pattern Recogn 1:275–282

    Google Scholar 

  4. Chen L, Hu R, Jiang J, Han Z (2014) Efficient learning based face hallucination approach via facial standard deviation prior. In: IEEE International Symposium on Circuits and Systems, pp 2057–2060

  5. Chen L, Hu R, Liang C, Li Q, Han Z (2017) A novel face super resolution approach for noisy images using contour feature and standard deviation prior. Multimed Tool Appl 76(2):2467–2493

    Article  Google Scholar 

  6. Dong X, Hu R, Jiang J, Han Z, Chen L, Gao G (2014) Noise face image hallucination via data-driven local eigentransformation. In: Pacific Rim Conference on Multimedia, pp 183–192

  7. Dong C, Loy CC, He K, Tang X (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Article  Google Scholar 

  8. Everingham M, Sivic J, Zisserman A (2006) Hello! My name is... Buffy–automatic naming of characters in TV video. In: British Machine Vision Conference, pp 899–908

  9. Gao W, Cao B, Shan S (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations[J]

  10. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image. International Conference on Computer Vision 349–356

  11. Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications. IEEE Trans Circ Syst Video Technol 21(10):1363–1377

    Article  Google Scholar 

  12. Jiang J, Hu R, Han Z, Huang K, Lu T (2013) Graph discriminant analysis on multi-manifold (GDAMM): a novel super-resolution method for face recognition. IEEE International Conference on Image Processing 1465–1468

  13. Jiang J, Hu R, Liang C, Han Z, Zhang C (2014) Face image super-resolution through locality-induced support regression. Signal Process 103:168–183

    Article  Google Scholar 

  14. Jiang J, Hu R, Wang Z, Han Z (2014) Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans Image Process 23(10):4220–4231

    Article  MathSciNet  Google Scholar 

  15. Jiang J, Hu R, Wang Z, Han Z (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281

    Article  Google Scholar 

  16. Lan C, Hu R, Huang K, Han Z (2010) Face hallucination with shape parameters projection constraint. In: Proceedings of the ACM international conference on Multimedia, pp 883–886

  17. Lee H, Battle A, Raina R, Ng AY (2007) Efficient sparse coding algorithms. In: Advances in neural information processing systems, pp 801–808

  18. Liu S, Yang MH (2014) Compressed face hallucination. In: IEEE International Conference on Image Processing, pp 4032–4036

  19. Liu C, Yuen J, Torralba A (2011) Sift flow: dense correspondence across scenes and its applications. IEEE Trans Pattern Anal Mach Intell 33(5):978–994

    Article  Google Scholar 

  20. Lucas B, Kanade T (1981) An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp 674–679

  21. Ma X, Zhang J, Qi C (2009) Position-based face hallucination method. In: ICME 2009 IEEE International Conference on Multimedia and Expo

  22. Qu S, Hu R, Chen S, Jiang J, Wang Z, Chen J (2014) Face hallucination via re-identified K-nearest neighbors embedding. In: IEEE International Conference on Multimedia and Expo, pp 1–6

  23. Rowley H, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38

    Article  Google Scholar 

  24. Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. Comput Vis Pattern Recogn 2, II:729–36

    Google Scholar 

  25. Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913

    Article  Google Scholar 

  26. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J Royal Stat Soc Series B (Methodological) 267–288

    MathSciNet  MATH  Google Scholar 

  27. Timofte R, De Smet V, Van Gool L (2013) Anchored neighborhood regression for fast example-based super-resolution. In: IEEE International Conference on Computer Vision, pp 1920–1927

  28. Wang X, Tang X (2005) Hallucinating face by eigentransformation . IEEE Trans Syst, Man, Cybern Part C (Applications and Reviews) 35(3):425–434

    Article  Google Scholar 

  29. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13 (4):600–612

    Article  Google Scholar 

  30. Yang J, Tang H, Ma Y, Huang T (2008) Face hallucination via sparse coding. In: IEEE International Conference on Image Processing, pp 1264–1267

  31. Yang J, Wright J, Huang TS, Ma Y (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  Google Scholar 

  32. Yang CY, Liu S, Yang MH (2013) Structured face hallucination. In: IEEE Conference on Computer Vision and Pattern Recognition, pp 1099–1106

  33. Zhou E, Fan H, Cao Z, Jiang Y, Yin Q (2015) Learning face hallucination in the wild. Assoc Advan Artif Intell 3871–3877

Download references

Acknowledgements

This work was supported in part by the National Nature Science Foundation of China under Grant No. 61901117, U1805262, in part by the Natural Science Foundation of Fujian Province under the Grant 2019J05060 and Grant 2019J01271, in part by the Fujian Provincial Education Department Project under Grant JT180094 and Grant JT180095, in part by the Hubei Key Laboratory of Intelligent Robot, Wuhan Institute of Technology, under Grant HBIR201906, in part by the Special Funds of the Central Government Guiding Local Science and Technology Development under Grant 2017L3009, and in part by the National Key Research and Development Program of China under Grant 2016YFB1001001.

Author information

Authors and Affiliations

  1. Fujian Provincial Engineering Technology Research Center of Photoelectric Sensing Application, Key Laboratory of OptoElectronic Science and Technology for Medicine of Ministry of Education, Fujian Provincial Key Laboratory of Photonics Technology, Fujian Normal University, Fuzhou, 350100, China

    Liang Chen

  2. The Hong Kong Polytechnic University, Hong Kong, People’s Republic of China

    Qing Li

  3. Harbin Institute of Technology, Harbin, People’s Republic of China

    Junjun Jiang

Authors
  1. Liang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junjun Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liang Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Li, Q. & Jiang, J. Noisy practical facial super-resolution method via deformable constrained model with small dataset. Multimed Tools Appl 79, 2577–2600 (2020). https://doi.org/10.1007/s11042-019-08277-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08277-7

Keywords

Search

Navigation