Skip to main content
SpringerLink
Log in

A novel face super resolution approach for noisy images using contour feature and standard deviation prior

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face Super Resolution (FSR) is to infer high resolution (HR) face images from given low resolution (LR) face images with the help of HR/LR training examples. The most representative FSR is NE methods, which are based on the consistency assumption that the HR/LR patch pairs form similar local geometric structures. But NE methods have difficulty in dealing with noisy facial images. The reason lies in the wrong neighborhood relationship caused by low quality scenarios that even two distinct patches have similar relation in local geometry. Therefore, the consistency assumption is not well held anymore. This paper presents a novel FSR approach suitable for noisy facial images. Our work are twofold. Firstly, different from the existing methods which directly enhance the noisy input image in intensity feature space, the proposed method introduces a contour feature which is robust to noise. By applying the contour feature as constraint, the noise effects can be effectively suppressed. Secondly, different from the existing methods which directly constrain the noisy input image with low quality contour feature, a standard deviation prior is proposed to enhance the low quality contour feature. Through enhancing the contour feature into high quality, the FSR reconstruction can be better constrained. Both simulation and the real-world scenario experiments demonstrate that the proposed approach outperforms most classic methods both quantitatively and qualitatively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183

    Article  Google Scholar 

  2. Bowyer KW, Chang K, Flynn P (2006) A survey of approaches and challenges in 3D and multi-modal 3D+ 2D face recognition. Computer vision and image understanding 101.1, pp 1–15

  3. Baker S, Kanade T (2000) Hallucinating faces[C]. In: Proceedings of the 4th IEEE international conference on automatic face and gesture recognition, 2000. IEEE, pp 83–88

  4. Baker S, Kanade T (2002) Limits on super-resolution and how to break them. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183

    Article  Google Scholar 

  5. Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising[C]. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 2. IEEE, pp 60–65

  6. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding[C]. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, 2004. CVPR 2004. IEEE, p 1: I-I

  7. Chen L, Hu R, Han Z et al (2013) A joint learning based face hallucination approach for low quality face image[C]. In: 20th IEEE international conference on image processing (ICIP), 2013. IEEE, p 972

  8. Chen L, Hu R, Jiang J et al (2014) Efficient learning based face hallucination approach via facial standard deviation prior[C]. In: IEEE international symposium on circuits and systems (ISCAS), 2014. IEEE, pp 2057–2060

  9. Chang KI, Bowyer KW, Flynn PJ (2005) An evaluation of multimodal 2D+ 3D face biometrics. In: IEEE Transactions on 27.4 Pattern Analysis and Machine Intelligence, pp 619–624

  10. Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel PCA-based prior. IEEE Trans Multimed 9(4):888–892

    Article  Google Scholar 

  11. Chan TM, Zhang J, Pu J et al (2009) Neighbor embedding based super-resolution algorithm through edge detection and feature selection. Pattern Recogn Lett 30(5):494–502

    Article  Google Scholar 

  12. Dong X, Hu R, Jiang J et al (2014) Noise face image hallucination via data-driven local eigentransformation[M]. In: Advances in multimedia information processing C PCM 2014. Springer International Publishing, pp 183–192

  13. Gao W, Cao B, Shan S et al (2008) The CAS-PEAL large-scale chinese face database and baseline evaluations. Humans IEEE Trans Syst Man, Cybern Part A Syst 38(1):149–161

    Article  Google Scholar 

  14. Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications. IEEE Trans Circ Syst Video Technol 21(10):1363–1377

    Article  Google Scholar 

  15. Jiang J, Hu R, Liang C et al (2014) Face image super-resolution through locality-induced support regression. Signal Process 103(10):168–183

    Article  Google Scholar 

  16. Jiang J, Hu R, Wang Z et al (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281

    Article  Google Scholar 

  17. Jiang J, Hu R, Liang C, Han Z, Zhang C (2014) Face image super-resolution through locality-induced support regression. Signal Process 103:168–183

    Article  Google Scholar 

  18. Jiang J, Hu R, Wang Z et al (2015) CDMMA: coupled discriminant multi-manifold analysis for matching low-resolution face images[J]. Signal Process

  19. Jiang J, Ma X, Cai Z et al (2015) Sparse support regression for image super-esolution[J]. Photonics J IEEE 7(5):1–11

    Article  Google Scholar 

  20. Jiang J, Hu R, Wang Z, Han Z, Ma J (2015) Facial image hallucination through coupled-layer neighbor embedding. IEEE Transactions on Circuits and Systems for Video Technology. doi:10.1109/TCSVT.2015.2433538

    Google Scholar 

  21. Kolouri S, Rohde GK (2015) Transport-based single frame super resolution of very low resolution face images[C]. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 4876–4884

  22. Lin Z, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation. IEEE Trans Pattern Anal Mach Intell 26(1):83–97

    Article  Google Scholar 

  23. Liu C, Shum HY, Zhang CS (2001) A two-step approach to hallucinating faces: global parametric model and local nonparametric model[C]. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001, CVPR 2001, vol 1. IEEE, pp 1: I-192-I-198

  24. Li B, Chang H, Shan S et al (2009) Aligning coupled manifolds for face hallucination. IEEE Signal Process Lett 16(11):957–960

    Article  Google Scholar 

  25. Lan C, Hu R, Huang K et al (2010) Face hallucination with shape parameters projection constraint[C]. In: Proceedings of the international conference on multimedia. ACM, pp 883–886

  26. Liu S, Yang M H (2014) Compressed face hallucination[C]. In: IEEE international conference on image processing (ICIP), 2014. IEEE, pp 4032–4036

  27. Li X, Orchard MT (2001) New edge-directed interpolation. IEEE Trans Image Process 10(10):1521–1527

    Article  Google Scholar 

  28. Ma X, Zhang J, Qi C (2009) Position-based face hallucination method[C]. In: IEEE international conference on multimedia and expo, ICME, 2009. IEEE, pp 290–293

  29. Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview. IEEE Signal Process Mag 20(3):21–36

    Article  Google Scholar 

  30. Park SW, Savvides M (2007) Breaking the limitation of manifold analysis for super-resolution of facial images[C]. Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE Int Conf I-573-I-576

  31. Rowley H, Baluja S, Kanade T (1998) Neural network-based face detection. IEEE Trans Pattern Anal Mach Intell 20(1):23–38

  32. Shao L, Yan R, Li X et al (2014) From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms. IEEE Trans Cybern 44(7):1001–1013

    Article  Google Scholar 

  33. Shao L, Zhang H, De HG (2008) An overview and performance evaluation of classification-based least squares trained filters. IEEE Trans Image Process 17 (10):1772–1782

    Article  MathSciNet  MATH  Google Scholar 

  34. Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, vol 2. IEEE, pp II-729

  35. Tan X, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650

    Article  MathSciNet  MATH  Google Scholar 

  36. Thomaz CE, Giraldi GA (2010) A new ranking method for principal components analysis and its application to face image analysis. Image Vis Comput 28(6):902–913

    Article  Google Scholar 

  37. Vezzetti E, Calignano F, Moos S (2010) Computer-aided morphological analysis for maxillo-facial diagnostic: a preliminary study. Journal of Plastic, Reconstructive, Aesthetic Surgery 63.2, pp 218–226

  38. Vezzetti E, Marcolin F, Stola V (2013) 3D human face soft tissues landmarking method: An advanced approach, Computers in Industry 64.9, pp 1326–1354

  39. Wang X, Tang X (2005) Hallucinating face by eigentransformation. IEEE Trans Syst Man Cybern Part C: Appl Rev 35(3):425–434

  40. Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  41. Yan R, Shao L, Liu Y (2013) Nonlocal hierarchical dictionary learning using wavelets for image denoising. IEEE Trans Image Process 22(12):4689–4698

    Article  MathSciNet  MATH  Google Scholar 

  42. Yang CY, Liu S, Yang MH (2013) Structured face hallucination[C]. In: IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 1099–1106

  43. Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    Article  MathSciNet  MATH  Google Scholar 

  44. Zou WWW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340. doi:10.1109/TIP.2011.2162432

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgments

The research is supported by the National Nature Science Foundation of China (No. 61231015, 61172173, 61201247, U1404618, 61303114), the National High Technology Research and Development Program of China (863 Program), Technology Research Program of Ministry of Public Security (No.2014JSYJA016), Guangdong-Hongkong Key Domain Breakthrough Project of China (No. 2012A090200007), The major Science and Technology Innovation Plan of Hubei Province(No. 2013AAA020), China Postdoctoral Science Foundation funded project (2013M530350), Specialized Research Fund for the Doctoral Program of Higher Education (No. 20130141120024), Key Technology RD Program of Wuhan(2013030409020109).

Author information

Authors and Affiliations

  1. National Engineering Research Center for Multimedia Software, School of Computer, Wuhan University, Wuhan, 430072, China

    Liang Chen, Ruimin Hu, Chao Liang & Zhen Han

  2. Department of Computer Science, City University of Hong Kong, Hong Kong, China

    Liang Chen & Qing Li

Authors
  1. Liang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruimin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chao Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhen Han
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruimin Hu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Hu, R., Liang, C. et al. A novel face super resolution approach for noisy images using contour feature and standard deviation prior. Multimed Tools Appl 76, 2467–2493 (2017). https://doi.org/10.1007/s11042-015-3145-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-015-3145-9

Keywords

Search

Navigation