Skip to main content
SpringerLink
Log in

Sign-correlation cascaded regression for face alignment

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Face alignment plays an important role in many applications such as face recognition and face reconstruction. Current regression based approaches can ease the multi-pose face alignment problem, but they fail to deal with the multiple local minima problem directly. To improve the performance of multi-pose facial landmark localization, in this paper we propose a sign correlation supervised descent method (SC-SDM) based on a nonlinear optimization theory. SC-SDM analyses the sign correlation between features and shapes and project both of them into a mutual sign-correlation subspace. By partitioning the whole multi-pose samples into a series of pose-consistent subsets, a group of models are learned from each subset. The experiments using the public multi-pose datasets has validated the partition and proved that SC-SDM can accurately separate samples into pose-consistent subsets, which reveals their latent relationships to pose. The comparison with state-of-the-art methods demonstrates that SC-SDM outperforms them, especially in uncontrolled conditions with various poses.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Asthana A, Zafeiriou S, Cheng S, Pantic M (2013) Robust discrim- inative response map fitting with constrained local models. IEEE Conference on Computer Vision and Pattern Recognition:3444–3451

  2. Belhumeur PN, Jacobs DW, Kriegman DJ, Kumar N (2013) Localizing parts of faces using a consensus of exemplars. IEEE Transactions on Pattern Analysis & Machine Intelligence 35(12):2930–2940

    Article  Google Scholar 

  3. Cao X, Wei Y, Wen F, Sun J (2012) Face alignment by explicit shape regression. US Patent App. 13/728,584

  4. Cheng D, Yang J, Wang J, Shi D, Liu X (2015) Double-noise-dual- problem approach to the augmented lagrange multiplier method for robust principal component analysis. Soft Comput 21:1–10

    Google Scholar 

  5. Cootes TF, Edwards GJ, Taylor CJ (2001) Active appearance models. Pattern Analysis & Machine Intelligence IEEE Transactions on 23(6):681–685

    Article  Google Scholar 

  6. Cootes TF, Taylor CJ, Cooper DH, Graham J (1995) Active shape models & their training and application. Computer Vision & Image Understanding 61(1):38–59

    Article  Google Scholar 

  7. Cristinacce D, Cootes TF (2006) Feature detection and tracking with constrained local models. BMVC 41:929–938

    MATH  Google Scholar 

  8. Dollar P, Welinder P, Perona P (2010) Cascaded pose regression. CVPR, IEEE Computer Society Conference 238(6):1078–1085

    Google Scholar 

  9. Dong S, Luo S (2016) Modified grey-level models for active shape model training. Conf Proc IEEE Eng Med Biol Soc 1:3791–3794

    Google Scholar 

  10. Gonzalezmora J, Torre FDL, Murthi R, Guil N, Zapata EL (2007) Bilinear active appearance models. IEEE International Conference on Computer Vision:1–8

  11. Gu L, Kanade T (2009) Face Alignment. Springer, Boston, MA

  12. Huang GB, Mattar M, Berg T, Learned-Miller E (2008) Labeled faces in the wild: A database for studying face recognition in unconstrained environments

  13. Kazemi V, Sullivan J (2014) One millisecond face alignment with an ensemble of regression trees. Computer Vision and Pattern Recognition:1867–1874

  14. Kostinger M, Wohlhart P, Roth PM, Bischof H (2011) Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization, vol 6-13. IEEE International Conference on Computer Vision Workshops, ICCV 2011 Workshops, Barcelona, pp 2144–2151

    Google Scholar 

  15. Le V, Brandt J, Lin Z, Bourdev L, Huang TS (2012) Interactive facial feature localization. European Conference on Computer Vision:679–692

  16. Lee HS, Kim D (2009) Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Transactions on Pattern Analysis & Machine Intelligence 31(6):1102–1116

    Article  Google Scholar 

  17. Ramanan D (2015) Face detection, pose estimation, and landmark localization in the wild. IEEE Conference on Computer Vision and Pattern Recognition:31–37

  18. Ren S, Cao X, Wei Y, Sun J (2014) Face alignment at 3000 fps via regressing local binary features. IEEE Trans Image Process:1685–1692

  19. Romdhani S (1999) A multi-view nonlinear active shape model using kernel pca. British Machine Vision Conference:483–492

  20. Sagonas C, Tzimiropoulos G, Zafeiriou S, Pantic M (2013) A semiautomatic methodology for facial landmark annotation. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 896–903

  21. Saragih JM, Lucey S, Cohn JF (2011) Deformable model fitting by regularized landmark mean-shift. Int J Comput Vis 91(2):200–215

    Article  MathSciNet  MATH  Google Scholar 

  22. Snchez-Lozano E, Martinez B, Valstar MF (2016) Cascaded regression with sparsified feature covariance matrix for facial landmark detection. Pattern Recogn Lett 73(C):19–25

    Article  Google Scholar 

  23. Wilks DS (2011) Canonical correlation analysis (cca). International Geophysics 100:563–582

    Article  Google Scholar 

  24. Xing J, Niu Z, Huang J, Hu W, Yan S (2014) Towards multi-view and partially-occluded face alignment. Computer Vision and Pattern Recognition:1829–1836

  25. Xiong X, De, la Torre F (2013) Supervised descent method and its applications to face alignment. IEEE Conference on Computer Vision & Pattern Recognition:532–539

  26. Xiong X, la Torre FD (2015) Global supervised descent method. IEEE Conference on Computer Vision and Pattern Recognition:2664–2673

  27. Yan J, Lei Z, Yi D, Li SZ (2013) Learn to combine multiple hypotheses for accurate face alignment. IEEE International Conference on Computer Vision Workshops:392–396

  28. Yang H, He X, Jia X, Patras I (2015) Robust face alignment under occlusion via regional predictive power estimation. IEEE Trans Image Process 24(8):2393–2403

    Article  MathSciNet  MATH  Google Scholar 

  29. Zhang Z, Luo P, Loy CC, Tang X (2014) Facial landmark detection by deep multi-task learning. European Conference on Computer Vision:94–108

  30. Zhang J, Shan S, Kan M, Chen X (2014) Coarse-to-fine auto-encoder networks (cfan) for real-time face alignment. In: Computer Vision ECCV 2014. Springer, pp. 1–16

  31. Zhang Y, Shi D, Gao J, Cheng D (2017) Low-rank-sparse subspace representation for robust regression. IEEE Conference on Computer Vision Pattern Recognition

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, Harbin, 150001, China

    Dansong Cheng, Yongqiang Zhang & Ce Liu

  2. School of Electrical Engineering and Automation, Harbin Institute of Technology, Harbin, 150001, China

    Xiaofang Liu

Authors
  1. Dansong Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongqiang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ce Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaofang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaofang Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cheng, D., Zhang, Y., Liu, C. et al. Sign-correlation cascaded regression for face alignment. Multimed Tools Appl 78, 26681–26699 (2019). https://doi.org/10.1007/s11042-019-7737-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-7737-7

Keywords

Search

Navigation