Skip to main content
SpringerLink
Log in

Face image super-resolution with pose via nuclear norm regularized structural orthogonal Procrustes regression

  • Cognitive Computing for Intelligent Application and Service
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

In real applications, the observed low-resolution face images usually have pose variations. Conventional learning-based methods ignore these variations; thus, the hallucinated high-resolution faces are not reasonable for the following recognition task. For recognition purpose, we prefer to obtain near-frontal faces. To this end, we propose a nuclear norm regularized structural orthogonal Procrustes regression (N2SOPR) approach in this work to acquire pose-robust feature representations for face hallucination with pose. The orthogonal Procrustes regression is used to seek an appropriate transformation between two data matrixes. Additionally, the nuclear norm regularization is imposed on the representation residual to preserve image structural property. We also impose a low-rank restraint on the combination weight to automatically cluster each input into the same subspace with the training samples. Both hallucination and recognition experiments conducted on common face databases have verified that our N2SOPR can obtain reasonable performance than some related methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Gao G, Yang J, Wu S et al (2015) Bayesian sample steered discriminative regression for biometric image classification. Appl Soft Comput 37:48–59

    Google Scholar 

  2. Huang P, Gao G (2016) Parameterless reconstructive discriminant analysis for feature extraction. Neurocomputing 190:50–59

    Google Scholar 

  3. Jing X-Y, Wu F, Zhu X et al (2016) Multi-spectral low-rank structured dictionary learning for face recognition. Pattern Recogn 59:14–25

    Google Scholar 

  4. Lai Z, Wong WK, Xu Y et al (2016) Approximate orthogonal sparse embedding for dimensionality reduction. IEEE Trans Neural Netw Learn Syst 27(4):723–735

    MathSciNet  Google Scholar 

  5. Mudunuri SP, Biswas S (2016) Low resolution face recognition across variations in pose and illumination. IEEE Trans Pattern Anal Mach Intell 38(5):1034–1040

    Google Scholar 

  6. Shen F, Shen C, Zhou X et al (2016) Face image classification by pooling raw features. Pattern Recogn 54:94–103

    Google Scholar 

  7. Tai Y, Yang J, Zhang Y et al (2016) Face recognition with pose variations and misalignment via orthogonal Procrustes regression. IEEE Trans Image Process 25(6):2673–2683

    MathSciNet  MATH  Google Scholar 

  8. Deng W, Hu J, Wu Z et al (2017) From one to many: pose-aware metric learning for single-sample face recognition. Pattern Recogn 77:426–437

    Google Scholar 

  9. Gao G, Yang J, Jing X-Y et al (2017) Learning robust and discriminative low-rank representations for face recognition with occlusion. Pattern Recogn 66:129–143

    Google Scholar 

  10. Yang J, Luo L, Qian J et al (2017) Nuclear norm based matrix regression with applications to face recognition with occlusion and illumination changes. IEEE Trans Pattern Anal Mach Intell 39(1):156–171

    Google Scholar 

  11. Yang M, Wang X, Zeng G et al (2017) Joint and collaborative representation with local adaptive convolution feature for face recognition with single sample per person. Pattern Recogn 66:117–128

    Google Scholar 

  12. Hamedani K, Seyyedsalehi SA, Ahamdi R (2016) Video-based face recognition and image synthesis from rotating head frames using nonlinear manifold learning by neural networks. Neural Comput Appl 27(6):1761–1769

    Google Scholar 

  13. Han B, He B, Sun T et al (2016) HSR: L 1/2-regularized sparse representation for fast face recognition using hierarchical feature selection. Neural Comput Appl 27(2):305–320

    Google Scholar 

  14. Zhu Y, Xue J (2017) Face recognition based on random subspace method and tensor subspace analysis. Neural Comput Appl 28(2):233–244

    Google Scholar 

  15. Wu F, Jing X-Y, Liu Q et al (2017) Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning. Neural Comput Appl 28(3):483–498

    Google Scholar 

  16. Lan R, Zhou Y, Tang YY (2017) Quaternionic weber local descriptor of color images. IEEE Trans Circuits Syst Video Technol 27(2):261–274

    Google Scholar 

  17. Zou WW, Yuen PC (2012) Very low resolution face recognition problem. IEEE Trans Image Process 21(1):327–340

    MathSciNet  MATH  Google Scholar 

  18. Freeman WT, Pasztor EC, Carmichael OT (2000) Learning low-level vision. Int J Comput Vis 40(1):25–47

    MATH  Google Scholar 

  19. Wang X, Tang X (2005) Hallucinating face by eigen transformation. IEEE Trans Syst Man Cybern Part C Appl Rev 35(3):425–434

    Google Scholar 

  20. Hu Y, Lam KM, Shen T et al (2011) A novel kernel-based framework for facial-image hallucination. Image Vis Comput 29(4):219–229

    Google Scholar 

  21. Shi J, Liu X, Qi C (2014) Global consistency, local sparsity and pixel correlation: a unified framework for face hallucination. Pattern Recogn 47(11):3520–3534

    Google Scholar 

  22. Huang H, He H, Fan X et al (2010) Super-resolution of human face image using canonical correlation analysis. Pattern Recogn 43(7):2532–2543

    MATH  Google Scholar 

  23. An L, Bhanu B (2014) Face image super-resolution using 2D CCA. Signal Process 103:184–194

    Google Scholar 

  24. Gao G, Yang J (2014) A novel sparse representation based framework for face image super-resolution. Neurocomputing 134:92–99

    Google Scholar 

  25. Wright J, Yang AY, Ganesh A et al (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Google Scholar 

  26. Bo C, Wang D (2015) A registration-based tracking algorithm based on noise separation. Optik-Int J Light Electron Opt 126(24):5806–5811

    Google Scholar 

  27. Li F, Lu H, Wang D et al (2016) Dual group structured tracking. IEEE Trans Circuits Syst Video Technol 26(9):1697–1708

    Google Scholar 

  28. Zhao W, Lu H, Wang D (2018) Multisensor image fusion and enhancement in spectral total variation domain. IEEE Trans Multimed 20(4):866–879

    Google Scholar 

  29. Chang H, Yeung D-Y, Xiong Y (2004) Super-resolution through neighbor embedding. In: Proceedings of IEEE computer society conference on computer vision and pattern recognition, pp 1275–1282

  30. Jiang J, Hu R, Wang Z et al (2014) Face super-resolution via multilayer locality-constrained iterative neighbor embedding and intermediate dictionary learning. IEEE Trans Image Process 23(10):4220–4231

    MathSciNet  MATH  Google Scholar 

  31. Jiang J, Hu R, Wang Z et al (2016) Facial image hallucination through coupled-layer neighbor embedding. IEEE Trans Circuits Syst Video Technol 26(9):1674–1684

    Google Scholar 

  32. Yang J, Wright J, Huang TS et al (2010) Image super-resolution via sparse representation. IEEE Trans Image Process 19(11):2861–2873

    MathSciNet  MATH  Google Scholar 

  33. Ma X, Zhang J, Qi C (2010) Hallucinating face by position-patch. Pattern Recogn 43(6):2224–2236

    Google Scholar 

  34. Jung C, Jiao L, Liu B et al (2011) Position-patch based face hallucination using convex optimization. IEEE Signal Process Lett 18(6):367–370

    Google Scholar 

  35. Wang Z, Hu R, Wang S et al (2014) Face hallucination via weighted adaptive sparse regularization. IEEE Trans Circuits Syst Video Technol 24(5):802–813

    Google Scholar 

  36. Jiang J, Hu R, Wang Z et al (2014) Noise robust face hallucination via locality-constrained representation. IEEE Trans Multimed 16(5):1268–1281

    Google Scholar 

  37. Jiang J, Ma J, Chen C et al (2017) Noise robust face image super-resolution through smooth sparse representation. IEEE Trans Cybern 47(11):3991–4002

    Google Scholar 

  38. Liu L, Chen CP, Li S et al (2018) Robust face hallucination via locality-constrained bi-layer representation. IEEE Trans Cybern 48(4):1189–1201

    Google Scholar 

  39. Dong C, Loy CC, He K et al (2016) Image super-resolution using deep convolutional networks. IEEE Trans Pattern Anal Mach Intell 38(2):295–307

    Google Scholar 

  40. Zhang Y, Li K, Li K et al (2018) Image super-resolution using very deep residual channel attention networks. arXiv preprint arXiv:1807.02758

  41. Zhang Y, Tian Y, Kong Y et al (2018) Residual dense network for image super-resolution. In: IEEE conference on computer vision and pattern recognition (CVPR), pp 2472–2481

  42. Kim S-J, Koh K, Lustig M et al (2007) An interior-point method for large-scale ℓ1-regularized least squares. IEEE J Sel Top Signal Process 1(4):606–617

    Google Scholar 

  43. Hurley JR, Cattell RB (1962) The Procrustes program: producing direct rotation to test a hypothesized factor structure. Behav Sci 7(2):258–262

    Google Scholar 

  44. Zhang F, Yang J, Tai Y et al (2015) Double nuclear norm-based matrix decomposition for occluded image recovery and background modeling. IEEE Trans Image Process 24(6):1956–1966

    MathSciNet  MATH  Google Scholar 

  45. Chen J, Yang J, Luo L et al (2015) Matrix variate distribution-induced sparse representation for robust image classification. IEEE Trans Neural Netw Learn Syst 26(10):2291–2300

    MathSciNet  Google Scholar 

  46. Cai JF, Candes EJ, Shen ZW (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    MathSciNet  MATH  Google Scholar 

  47. Phillips PJ, Wechsler H, Huang J et al (1998) The FERET database and evaluation procedure for face-recognition algorithms. Image Vis Comput 16(5):295–306

    Google Scholar 

  48. Wang Z, Bovik AC, Sheikh HR et al (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Google Scholar 

Download references

Acknowledgements

This work was partially supported by the National Natural Science Foundation of China under Grant Nos. 61502245, 61772568, 61571236, 61473086, the Natural Science Foundation of Jiangsu Province under Grant No. BK20150849, the Fundamental Research Funds for the Central Universities of China (No. 18lgzd15), Open Fund Project of Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education (Nanjing University of Science and Technology) (No. JYB201709).

Author information

Authors and Affiliations

  1. Institute of Advanced Technology, Nanjing University of Posts and Telecommunications, Nanjing, People’s Republic of China

    Guangwei Gao, Dong Zhu & Hao Gao

  2. School of Data and Computer Science, Sun Yat-Sen University, Guangzhou, People’s Republic of China

    Meng Yang

  3. Department of Mechanical and Control Engineering, Kyushu Institute of Technology, Kitakyushu, Japan

    Huimin Lu

  4. School of Automation, Southeast University, Nanjing, People’s Republic of China

    Wankou Yang

  5. Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information of Ministry of Education, Nanjing University of Science and Technology, Nanjing, People’s Republic of China

    Guangwei Gao

Authors
  1. Guangwei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Meng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huimin Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wankou Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hao Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangwei Gao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, G., Zhu, D., Yang, M. et al. Face image super-resolution with pose via nuclear norm regularized structural orthogonal Procrustes regression. Neural Comput & Applic 32, 4361–4371 (2020). https://doi.org/10.1007/s00521-018-3826-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-3826-1

Keywords

Search

Navigation