Skip to main content
SpringerLink
Log in

Face super resolution based on parent patch prior for VLQ scenarios

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face Super Resolution (FSR) is to infer High Resolution (HR) facial images from given Low Resolution (LR) ones with the assistance of LR and HR training pairs. Among existing methods, Neighbor Embedding(NE) FSR methods are superior in visual and objective quality than holistic based methods. These NE methods are based on the consistency assumption that the neighbors in HR/LR space form similar local geometry. But when LR images are in Very Low Quality (VLQ), the LR patches are seriously contaminated that even two distinct patches form similar appearance, which means that the consistency assumption is not well held anymore. To solve this problem, in this paper we use the target patch as well as the surrounding pixels, which we call parent patch, to represent the target patch. By incorporating the peripheral information, the parent patch is much more robust to noise in the LR and HR consistency learning. The effectiveness of proposed method is verified both quantitatively and qualitatively. In this paper, we also discuss the boundary and the paradox of the multi-scaled parent patch prior in NE based FSR framework.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Baker S, Kanade T (2002) Limits on super-resolution and how to break them[J]. IEEE Trans Pattern Anal Mach Intell 24(9):1167–1183

    Article  Google Scholar 

  2. Baker S, Kanade T (2000) Hallucinating faces[C]. In: Proceedings. Fourth IEEE International Conference on Automatic Face and Gesture Recognition, 2000. IEEE, pp 83–88

  3. Chang H, Yeung DY, Xiong Y (2004) Super-resolution through neighbor embedding[C]. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004. IEEE, pp 1: I-I

  4. Chakrabarti A, Rajagopalan AN, Chellappa R (2007) Super-resolution of face images using kernel PCA-based prior[J]. IEEE Trans Multimed 9(4):888–892

    Article  Google Scholar 

  5. Dong X, Hu R, Jiang J et al (2014) Noise Face Image Hallucination via Data-Driven Local Eigentransformation[M]. In: Advances in Multimedia Information Processing C PCM 2014Springer International Publishing, pp 183–192

  6. Glasner D, Bagon S, Irani M (2009) Super-resolution from a single image[C]. In: 2009 IEEE 12th International Conference on Computer Vision. IEEE, pp 349–356

  7. Gao W, Cao B, Shan S et al (2008) The CAS-PEAL large-scale Chinese face database and baseline evaluations[J]. IEEE Tran Syst Man Cybern Part A Syst Human 38(1):149–161

    Article  Google Scholar 

  8. Huang H, Wu N (2011) Fast Facial Image Super-Resolution via Local Linear Transformations for Resource-Limited Applications[J]. IEEE Trans Circ Syst Video Technol 21(10):1363–1377

    Article  MathSciNet  Google Scholar 

  9. Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications[J]. IEEE Trans Circ Syst Video Technol 21(10):1363–1377

    Article  MathSciNet  Google Scholar 

  10. Huang H, Wu N (2011) Fast facial image super-resolution via local linear transformations for resource-limited applications[J]. IEEE Trans Circ Syst Video Technol 21(10):1363–1377

    Article  MathSciNet  Google Scholar 

  11. Jiang J, Hu R, Liang C et al (2014) Face image super-resolution through locality-induced support regression[J]. Signal Process 103(10):168–183

    Article  Google Scholar 

  12. Jiang J, Hu R, Wang Z et al (2014) Noise robust face hallucination via locality-constrained representation[J]. IEEE Trans Multimed 16(5):1268–1281

    Article  Google Scholar 

  13. Jiang J, Hu R, Wang Z et al (2015) CDMMA: Coupled discriminant multi-manifold analysis for matching low-resolution face images[J]

  14. Jiang J, Ma X, Cai Z et al (2015) Sparse Support Regression for Image Super-Resolution[J]. IEEE Photon J 7(5):1–11

    Article  Google Scholar 

  15. Jiang J, Hu R, Liang C, Han Z, Zhang C (2014) Face image super-resolution through locality-induced support regression. Signal Process 103:168–183

    Article  Google Scholar 

  16. Jiang J, Hu R, Wang Z, Han Z, Ma J Facial Image Hallucination through Coupled-Layer Neighbor Embedding

  17. Kolouri S, Rohde GK (2015) Transport-based single frame super resolution of very low resolution face images[C]. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 4876–4884

  18. Lin Z, Shum HY (2004) Fundamental limits of reconstruction-based superresolution algorithms under local translation[J]. IEEE Trans Pattern Anal Mach Intell 26(1):83–97

    Article  Google Scholar 

  19. Liu C, Shum HY, Zhang CS (2001) A two-step approach to hallucinating faces: global parametric model and local nonparametric model[C]. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2001. CVPR 2001, vol 1. IEEE, pp 1: I-192-I-198

  20. Li B, Chang H, Shan S et al (2009) Aligning coupled manifolds for face hallucination[J]. IEEE Signal Process Lett 16(11):957–960

    Article  Google Scholar 

  21. Lan C, Hu R, Huang K et al (2010) Face hallucination with shape parameters projection constraint[C]. In: Proceedings of the international conference on Multimedia. ACM, pp 883–886

  22. Liu S, Yang MH (2014) Compressed face hallucination[C]. In: 2014 IEEE International Conference on Image Processing (ICIP). IEEE, pp 4032–4036

  23. Ma X, Zhang J, Qi C (2009) Position-based face hallucination method[C]. In: IEEE International Conference on Multimedia and Expo, 2009. ICME 2009. IEEE, pp 290–293

  24. Park SC, Park MK, Kang MG (2003) Super-resolution image reconstruction: a technical overview[J]. IEEE Signal Process Mag 20(3):21–36

    Article  Google Scholar 

  25. Peng C, Gao X, Wang N et al (2015) Multiple Representations-Based Face Sketch-Photo Synthesis[J]

  26. Shao L, Yan R, Li X et al (2014) From heuristic optimization to dictionary learning: a review and comprehensive comparison of image denoising algorithms[J]. IEEE Trans Cybern 44(7):1001–1013

    Article  Google Scholar 

  27. Shao L, Zhang H, De Haan G (2008) An overview and performance evaluation of classification-based least squares trained filters[J]. IEEE Trans Image Process 17 (10):1772–1782

    Article  MathSciNet  Google Scholar 

  28. Sun J, Zheng NN, Tao H, Shum HY (2003) Image hallucination with primal sketch priors. In: Proceedings. 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003, vol 2. IEEE, pp II-729

  29. Su K, Tian Q, Xue Q et al (2005) Neighborhood issue in single-frame image super-resolution[C]. In: IEEE International Conference on Multimedia and Expo, 2005. ICME 2005. IEEE, p 4

  30. Wang N, Tao D, Gao X et al (2014) A comprehensive survey to face hallucination[J]. Int J Comput Vis 106(1):9–30

    Article  Google Scholar 

  31. Wang X, Tang X (2005) Hallucinating face by eigentransformation[J]. IEEE Trans Syst Man Cybern Part C Appl Rev 35(3):425–434

    Article  Google Scholar 

  32. Wang N, Li J, Tao D, Li X, Gao X (2013) Heterogeneous image transformation. Pattern Recogn Lett 34(1):77–84

    Article  Google Scholar 

  33. Wang N, Tao D, Gao X, Li X, Li J (2013) Transductive face sketch-photo synthesis. IEEE Trans Neural Netw Learn Syst 24(9):1364–1376

    Article  Google Scholar 

  34. Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity[J]. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  35. Wang Z, Hu R, Wang S et al (2014) Face hallucination via weighted adaptive sparse regularization[J]. IEEE Trans Circ Syst Video Technol 24(5):802–813

    Article  Google Scholar 

  36. Yan R, Shao L, Liu Y (2013) Nonlocal hierarchical dictionary learning using wavelets for image denoising[J]. IEEE Trans Image Process 22(12):4689–4698

    Article  MathSciNet  Google Scholar 

  37. Yang CY, Liu S, Yang MH (2013) Structured face hallucination[C]. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, pp 1099–1106

  38. Yang J, Tang H, Ma Y et al (2008) Face hallucination via sparse coding[C]. In: 15th IEEE International Conference on Image Processing, 2008. ICIP 2008. IEEE, pp 1264–1267

  39. Zou WWW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327,340. doi:10.1109/TIP.2011.2162432

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgments

The research is supported by the National High Technology Research and Development Program of China (863 Program No. 2015AA016306); the National Nature Science Foundation of China (No. 61231015, 61172173, 61303114, U1404618, 61501413, 61502354); the Internet of Things Development Funding Project of Ministry of industry in 2013(No. 25); the Technology Research Program of Ministry of Public Security (No. 2014JSYJA016); the China Postdoctoral Science Foundation funded project (2013M530350, 2014M562058); the Specialized Research Fund for the Doctoral Program of Higher Education (No. 20130141120024); the Fundamental Research Funds for the Central Universities(2042014kf0025). This work was partly supported by the EU FP7 QUICK project under Grant Agreement No. PIRSES-GA-2013-612652. The authors would like to thank Dr. Junjun Jiang for his important contribution in this paper.

Author information

Authors and Affiliations

  1. State Key Laboratory of Software Engineering, School of Computer of Wuhan University, Wuhan, People’s Republic of China

    Liang Chen, Ruimin Hu & Zhen Han

  2. Collaborative Innovation Center of Geospatial Technology, Wuhan, People’s Republic of China

    Liang Chen, Ruimin Hu & Zhen Han

  3. NERCMS, Computer School of Wuhan University, Wuhan, People’s Republic of China

    Liang Chen, Ruimin Hu & Zhen Han

  4. MERC, Department of Computer Science, City University of Hong Kong, Hong Kong, People’s Republic of China

    Liang Chen, Qing Li & Zheng Lu

Authors
  1. Liang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruimin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhen Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zheng Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ruimin Hu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, L., Hu, R., Han, Z. et al. Face super resolution based on parent patch prior for VLQ scenarios. Multimed Tools Appl 76, 10231–10254 (2017). https://doi.org/10.1007/s11042-016-3611-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3611-z

Keywords

Search

Navigation