Skip to main content
Springer Nature Link
Log in

Faster Real-Time Face Alignment Method on CPU

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2019)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 11857))

Included in the following conference series:

Abstract

Face alignment for facial images captured in-the-wild is a challenging and important problem. In this work, we introduced a two-stage face alignment method in order to solve the problem of the normal face alignment method running slowly on the CPU. Using the residual error between ground truth and mean shape as a training label makes the network easier to converge. The joint input of heatmap and original data in the second stage deepens the feature learning of these landmarks, making the minimal network also has suitable performance. The convolution and pooling structure allow the network to be faster and have good learning ability. The test results on open datasets show that our method has a significant improvement in processing performance with real-time CPU speed of 1100 fps while maintaining high accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Bhagavatula, C., Zhu, C., Luu, K., Savvides, M.: Faster Than Real-time Facial Alignment: A 3D Spatial Transformer Network Approach in Unconstrained Poses. arXiv:1707.05653 [cs] (2017)

  2. Tang, F., Zhang, J., Feng, Y., Guan, Q., Zhou, X.: Real-time face alignment enhancement by tracking. In: 2016 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 1011–1016. IEEE, Qingdao (2016). https://doi.org/10.1109/ROBIO.2016.7866457

  3. Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 60(2), 135–164 (2014)

    Google Scholar 

  4. Xiong, X., De la Torre, F.: Supervised descent method and its applications to face alignment. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 532–539. IEEE, Portland (2013). https://doi.org/10.1109/CVPR.2013.75

  5. Xiao, S., Feng, J., Xing, J., Lai, H., Yan, S., Kassim, A.: Robust facial landmark detection via recurrent attentive-refinement networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 57–72. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_4

    Chapter  Google Scholar 

  6. Ren, S., Cao, X., Wei, Y., Sun, J.: Face alignment at 3000 FPS via regressing local binary features. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1685–1692. IEEE, Columbus (2014). https://doi.org/10.1109/CVPR.2014.218

  7. Lee, D., Park, H., Yoo, C.D.: Face alignment using cascade Gaussian process regression trees. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4204–4212. IEEE, Boston (2015). https://doi.org/10.1109/CVPR.2015.7299048

  8. Zhu, S., Li, C., Loy, C.C., Tang, X.: Face alignment by coarse-to-fine shape searching. In: 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4998–5006. IEEE, Boston (2015). https://doi.org/10.1109/CVPR.2015.7299134

  9. Wu, W., Qian, C., Yang, S., Wang, Q., Cai, Y., Zhou, Q.: Look at Boundary: A Boundary-Aware Face Alignment Algorithm. arXiv:1805.10483 [cs] (2018)

  10. Kowalski, M., Naruniec, J., Trzcinski, T.: Deep Alignment Network: A convolutional neural network for robust face alignment. arXiv:1706.01789 [cs] (2017)

  11. Milborrow, S., Nicolls, F.: Locating facial features with an extended active shape model. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5305, pp. 504–513. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88693-8_37

    Chapter  Google Scholar 

  12. Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: 2013 IEEE International Conference on Computer Vision Workshops, pp. 397–403. IEEE, Sydney (2013). https://doi.org/10.1109/ICCVW.2013.59

  13. Saragih, J., Goecke, R.: A nonlinear discriminative approach to AAM fitting. In: 2007 IEEE 11th International Conference on Computer Vision, pp. 1–8. IEEE, Rio de Janeiro (2007). https://doi.org/10.1109/ICCV.2007.4409106

  14. Saragih, J.M., Lucey, S., Cohn, J.F.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. 91, 200–215 (2011). https://doi.org/10.1007/s11263-010-0380-4

    Article  MathSciNet  MATH  Google Scholar 

  15. Yang, S., Luo, P., Loy, C.C., Tang, X.: From Facial Parts Responses to Face Detection: A Deep Learning Approach. arXiv:1509.06451 [cs] (2015)

  16. Smith, B.M., Zhang, L.: Collaborative facial landmark localization for transferring annotations across datasets. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8694, pp. 78–93. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10599-4_6

    Chapter  Google Scholar 

  17. Sun, Y., Wang, X., Tang, X.: Deep convolutional network cascade for facial point detection. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3476–3483. IEEE, Portland (2013). https://doi.org/10.1109/CVPR.2013.446

  18. Sun, Y., Wang, X., Tang, X.: Deep learning face representation from predicting 10,000 classes. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1891–1898. IEEE, Columbus (2014). https://doi.org/10.1109/CVPR.2014.244

  19. Sun, Y., Wang, X., Tang, X.: Hybrid deep learning for face verification. In: TPAMI (2016)

    Google Scholar 

  20. Trigeorgis, G., Snape, P., Nicolaou, M.A., Antonakos, E., Zafeiriou, S.: Mnemonic descent method: a recurrent process applied for end-to-end face alignment. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4177–4187. IEEE, Las Vegas (2016). https://doi.org/10.1109/CVPR.2016.453

  21. Wu, W., Yang, S.: Leveraging intra and inter-dataset variations for robust face alignment. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 2096–2105. IEEE, Honolulu (2017). https://doi.org/10.1109/CVPRW.2017.261

  22. Wu, Y., Gou, C., Ji, Q.: Simultaneous facial landmark detection, pose and deformation estimation under facial occlusion. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5719–5728. IEEE, Honolulu (2017). https://doi.org/10.1109/CVPR.2017.606

  23. Zhu, S., Li, C., Loy, C.C., Tang, X.: Unconstrained face alignment via cascaded compositional learning. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3409–3417. IEEE, Las Vegas (2016). https://doi.org/10.1109/CVPR.2016.371

  24. Liu, Y., Jourabloo, A., Ren, W., Liu, X.: Dense face alignment. In: 2017 IEEE International Conference on Computer Vision Workshops (ICCVW), pp. 1619–1628. IEEE, Venice (2017). https://doi.org/10.1109/ICCVW.2017.190

  25. Zhu, Z., Luo, P., Wang, X., Tang, X.: Deep learning identity-preserving face space. In: ICCV (2013)

    Google Scholar 

  26. Burgos-Artizzu, X.P., Perona, P., Dollar, P.: Robust face landmark estimation under occlusion. In: 2013 IEEE International Conference on Computer Vision, pp. 1513–1520. IEEE, Sydney (2013). https://doi.org/10.1109/ICCV.2013.191

  27. Ghiasi, G., Fowlkes, C.C.: Occlusion coherence: localizing occluded faces with a hierarchical deformable part model. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1899–1906. IEEE, Columbus (2014). https://doi.org/10.1109/CVPR.2014.306

  28. Zhang, Z., Luo, P., Loy, C.C., Tang, X.: Learning deep representation for face alignment with auxiliary attributes. IEEE Trans. Pattern Anal. Mach. Intell. 38, 918–930 (2016). https://doi.org/10.1109/TPAMI.2015.2469286

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Cognitive Computing Technology Joint Laboratory, Wave Group, Beijing, 100083, China

    Pengfei Duan, Xin Ning, Yuan Shi, Shaolin Zhang & Weijun Li

  2. Laboratory of Artificial Neural Networks and High-Speed Circuits, Institute of Semiconductors, Chinese Academy of Sciences, Beijing, 100083, China

    Xin Ning & Weijun Li

Authors
  1. Pengfei Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xin Ning
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuan Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shaolin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weijun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xin Ning .

Editor information

Editors and Affiliations

  1. School of EECS, Peking University, Beijing, China

    Zhouchen Lin

  2. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Liang Wang

  3. Nanjing University, Nanjing, Jiangsu, China

    Jian Yang

  4. Xidian University, Xi'an, China

    Guangming Shi

  5. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  6. Institute of Artificial Intelligence, Xi'an Jiaotong University, Xi'an, Shaanxi, China

    Nanning Zheng

  7. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  8. Northwestern Polytechnical University, Xi'an, China

    Yanning Zhang

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Duan, P., Ning, X., Shi, Y., Zhang, S., Li, W. (2019). Faster Real-Time Face Alignment Method on CPU. In: Lin, Z., et al. Pattern Recognition and Computer Vision. PRCV 2019. Lecture Notes in Computer Science(), vol 11857. Springer, Cham. https://doi.org/10.1007/978-3-030-31654-9_34

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-31654-9_34

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-31653-2

  • Online ISBN: 978-3-030-31654-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics