Skip to main content
SpringerLink
Log in

Advances and Trends in Video Face Alignment

  • Chapter
  • First Online:
Recent Advances in Computer Vision

Part of the book series: Studies in Computational Intelligence ((SCI,volume 804))

Abstract

Face alignment in a video is an important research area in computer vision and can provides strong support for video face recognition, face animation, etc. It is different from face alignment in a single image where each face is regarded as an independent individual. For the latter, lack of amount of information makes the face alignment an under-determined problem although good results have been obtained by using prior information and auxiliary models. For the former, temporal and spatial relations are among faces in a video. These relations can impose constraints among multiple face images each other and help to improve alignment performance. In the chapter, definition of face alignment in a video and its significance are described. Methods for face alignment in a video are divided into three kinds: face alignment using image alignment algorithms, joint alignment of face images, and face alignment using temporal and spatial continuities. The first kind of face alignment is studied and some of surveys have described the work. The chapter will mainly focus on joint face alignment and face alignment using temporal and spatial continuities. Herein, some representative methods are described, and some factors influencing alignment performance are analyzed. Then the state-of-the-art methods are described and the future trends of face alignment in a video are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Gang, Z., Jingsheng, C., Ya, S., Hassaballah, M., Lianqiang, N.: Advances in Video Face Recognition. Science Press, China (2018). ISBN 9787030538468

    Google Scholar 

  2. Hassaballah, M., Saleh, A.: Face recognition: Challenges, achievements and future directions. IET Comput. Vis. J. 9(4), 614–626 (2015)

    Article  Google Scholar 

  3. Shan, S.G., Gao, W., Chang, Y.Z., Cao, B., Chen, X.L.: Curse of mis-alignment problem in face recognition. Chin. J. Comput. 28(5), 782–791 (2005)

    Google Scholar 

  4. Wagner, A., Wright, J., Ganesh, A., Zhou, Z.H., Mobahi, H., Ma, Y.: Towards a practical face recognition system: robust alignment and illumination by sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 34(2), 372–386 (2012)

    Article  Google Scholar 

  5. Lucas, B.D., Kanade, T.: An iterative image registration technique with an application to stereo vision. In: International Joint Conference on Artificial Intelligence, pp. 674–679. Vancouver, Can (1981)

    Google Scholar 

  6. Hager, G.D., Belhumeur, P.N.: Efficient region tracking with parametric models of geometry and illumination. IEEE Trans. Pattern Anal. Mach. Intell. 20(10), 1025–1039 (1998)

    Article  Google Scholar 

  7. Brown, L.G.: A survey of image registration techniques. ACM Comput. Surv. 24(4), 325–376 (1992)

    Article  Google Scholar 

  8. Zitová, B., Flusser, J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003)

    Article  Google Scholar 

  9. Jin, X., Tan, X.Y.: Face alignment in-the-wild: a survey. Comput. Vis. Image Underst. 162, 1–22 (2017)

    Article  Google Scholar 

  10. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

    Article  Google Scholar 

  11. Cristinacce, D., Cootes, T.: Automatic feature localization with constrained local models. Pattern Recogn. 41(10), 3054–3067 (2008)

    Article  Google Scholar 

  12. Gao, X.B., Su, Y., Li, X.L., Tao, D.C.: A review of active appearance models. IEEE Trans. Syst. Man Cybern. Part C-Appl. Rev. 40(2), 145–158 (2010)

    Article  Google Scholar 

  13. Xing, J.L., Niu, Z.H., Huang, J.S.: Towards robust and accurate multi-view and partially-occluded face alignment. IEEE Trans. Pattern Anal. Mach. Intell. 40(4), 987–1001 (2018)

    Article  Google Scholar 

  14. Tzimiropoulos, G., Pantic, M.: Fast algorithms for fitting active appearance models to unconstrained images. Int. J. Comput. Vis. 122(1), 17–33 (2017)

    Article  MathSciNet  Google Scholar 

  15. Saragih, J.M., Lucey, S., Cohn, J.E.: Deformable model fitting by regularized landmark mean-shift. Int. J. Comput. Vis. 91(2), 200–215 (2011)

    Article  MathSciNet  Google Scholar 

  16. Felzensawalb, P.F., Huttenlocher, D.P.: Pictorial structures for object recognition. Int. J. Comput. Vis. 61(1), 55–79 (2005)

    Article  Google Scholar 

  17. Zhou, F., Brandt, J., Lin, Z.: Exemplar-based graph matching for robust facial landmark localization. In: IEEE International Conference on Computer Vision, pp. 1025–1032. Sydney, Australia, 1–8 December 2013

    Google Scholar 

  18. Li, H.S., Huang, X.L., He, L.: Object matching using a locally affine invariant and linear programming techniques. IEEE Trans. Pattern Anal. Mach. Intell. 35(2), 411–424 (2013)

    Article  Google Scholar 

  19. Zhu, X.X., Ramanan, D.: Face detection, pose estimation, and landmark localization in the wild. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2879–2886. Providence, USA, 16–21 June 2012

    Google Scholar 

  20. Belhumeur, P.N., Jacobs, D.W., Kriegman, D.J., Kumar, N.: Localizing parts of faces using a consensus of exemplars. IEEE Trans. Pattern Anal. Mach. Intell. 35(12), 2930–2940 (2013)

    Article  Google Scholar 

  21. Learned-Miller, E.G.: Data driven image models through continuous joint alignment. IEEE Trans. Pattern Anal. Mach. Intell. 28(2), 236–250 (2006)

    Article  Google Scholar 

  22. Gross, R., Matthews, I., Baker, S.: Generic vs. person specific active appearance models. Image Vis. Comput. 23(12), 1080–1093 (2005)

    Article  Google Scholar 

  23. Cootes, T.F., Twining, C.J., Petrovic, V.S., Babalola, K.O., Taylor, C.J.: Computing accurate correspondences across groups of images. IEEE Trans. Pattern Anal. Mach. Intell. 32(11), 1994–2005 (2010)

    Article  Google Scholar 

  24. Marsland, S., Twining, C.J., Taylor, C.J.: A minimum description length objective function for groupwise non-rigid image registration. Image Vis. Comput. 26(3), 333–346 (2008)

    Article  Google Scholar 

  25. Sidorov, K.A., Richmond, S., Marshall, D.: Efficient groupwise non-rigid registration of textured surfaces. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2401–2408. Colorado Springs, USA, 20–25 June 2011

    Google Scholar 

  26. Basri, R., Jacobs, D.W.: Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. 25(2), 218–233 (2003)

    Article  Google Scholar 

  27. Zhao, C., Cham, W.K., Wang, X.G.: Joint face alignment with a generic deformable face model. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 561–568. Colorado Springs, USA, 20–25 June 2011

    Google Scholar 

  28. Smith, B.M., Zhang, L.: Joint face alignment with non-parametric shape models. In: European Conference on Computer Vision, pp. 43–56. Florence, Italy, 7–13 October 2012

    Chapter  Google Scholar 

  29. Irani, M., Peleg, S.: Super resolution from image sequences. In: International Conference on Pattern Recognition, pp. 115–120. Atlantic City, USA, 16–21 June 1990

    Google Scholar 

  30. Horn, B.K.P., Schunck, B.G.: Determining optical flow. Artif. Intell. 17, 185–203 (1981)

    Article  Google Scholar 

  31. Liu, C., Yuen, J., Torralba, A.: Sift flow: dense correspondence across scenes and its applications. IEEE Trans. Pattern Anal. Mach. Intell. 33(5), 978–994 (2011)

    Article  Google Scholar 

  32. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  MathSciNet  Google Scholar 

  33. Dowson, N.D.H., Bowden, R.: Simultaneous modeling and tracking (SMAT) of feature sets. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 99–105. San Diego, USA, 20–25 June 2005

    Google Scholar 

  34. Sung, J., Kanade, T., Kim, D.: Pose robust face tracking by combining active appearance models and cylinder head models. Int. J. Comput. Vis. 80(2), 260–274 (2008)

    Article  Google Scholar 

  35. Kahraman, F., Gokmen, M., Darkner, S., Larsen, R.: An active illumination and appearance (AIA) model for face alignment. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3616–3622. Minneapolis, USA, 17–22 June 2007

    Google Scholar 

  36. Roh, M.C., Oguri, T., Kanade, T.: Face alignment robust to occlusion. In: IEEE International Conference on Automatic Face & Gesture Recognition, pp. 239–244. Santa Barbara, USA, 21–25 March 2011

    Google Scholar 

  37. Dantone, M., Gall, J., Fanelli, G., Gool, L.V.: Real-time facial feature detection using conditional regression forests. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2578–2585. Providence, USA, 16–21 June 2012

    Google Scholar 

  38. Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 177–190 (2014)

    Article  MathSciNet  Google Scholar 

  39. Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874. Columbus, USA, 23–28 June 2014

    Google Scholar 

  40. Ren, S.Q., Cao, X.D., Wei, Y.C., Sun, J.: Face alignment at 3000 fps via regressing local binary features. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1685–1692. Columbus, USA, 23–28 June 2014

    Google Scholar 

  41. Lee, H.S., Kim, D.: Tensor-based AAM with continuous variation estimation: application to variation-robust face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 31(6), 1102–1116 (2009)

    Article  Google Scholar 

  42. Shi, B.G., Bai, X., Liu, W.Y., Wang, J.D.: Face alignment with deep regression. IEEE Trans. Neural Netw. Learn. Syst. 29(1), 183–194 (2018)

    Article  MathSciNet  Google Scholar 

  43. Lv, J.J., Shao, X.H., Xing, J.L., Cheng, C., Zhou, X.: A deep regression architecture with two-stage re-initialization for high performance facial landmark detection. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3691–3700. Honolulu, USA, 21–26 July 2017

    Google Scholar 

  44. Dibeklioglu, H., Salah, A.A., Gevers, T.: A statistical method for 2-D facial landmarking. IEEE Trans. Image Process. 21(2), 844–858 (2012)

    Article  MathSciNet  Google Scholar 

  45. Hu, C.B., Xiao, J., Matthews, I., Baker, S., Cohn, J., Kanade, T.: Fitting a single active appearance model simultaneously to multiple images. In: British Machine Vision Conference, pp. 437–446. London, UK, 7–9 September 2004

    Google Scholar 

  46. Su, Y.C., Ai, H.Z., Lao, S.H.: Multi-view face alignment using 3D shape model for view estimation. In: 3rd IAPR/IEEE International Conference on Advances in Biometrics, pp. 179–188. Alghero, Italy, 2–5 June 2009

    Chapter  Google Scholar 

  47. Anderson, R., Stenger, B., Cipolla, R.: Using bounded diameter minimum spanning trees to build dense active appearance models. Int. J. Comput. Vis. 110(1), 48–57 (2014)

    Article  Google Scholar 

  48. Bolkart, T., Wuhrer, S.: A groupwise multilinear correspondence optimization for 3D faces. In: IEEE International Conference on Computer Vision, pp. 3604–3612. Santiago, Chile, 11–18 December 2015

    Google Scholar 

  49. Liu, D., Nocedal, J.: On the limited memory method for large scale optimization. Math. Prog. Ser. A B 45(1), 503–528 (1989)

    Article  MathSciNet  Google Scholar 

  50. Matthews, I., Ishikawa, T., Baker, S.: The template update problem. IEEE Trans. Pattern Anal. Mach. Intell. 26(6), 810–815 (2004)

    Article  Google Scholar 

  51. Huang, C., Ding, X.Q., Fang, C.: Pose robust face tracking by combing view-based AAMs and temporal filters. Comput. Vis. Image Underst. 116(7), 777–792 (2012)

    Article  Google Scholar 

  52. Liu, X.M.: Video-based face model fitting using adaptive active appearance model. Image Vis. Comput. 28(7), 1162–1172 (2010)

    Article  Google Scholar 

  53. Zhang G., Tang S.K., Li J.Q.: Face landmark point tracking using LK pyramid optical flow. In: Tenth International Conference on Machine Vision. Vienna, Austria, 13–15 November 2018

    Google Scholar 

  54. Hassaballah, M., Bekhet, S., Amal A.M.R., Gang, Z.: Facial features detection and localization. In: Recent Advances in Computer Vision—Theories and Applications. Studies in Computational Intelligence Series, Springer, 2019

    Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant 61372176. It was also supported by the Liaoning Province Science and Technology Department of China under Grant 201602552.

Author information

Authors and Affiliations

  1. School of Software, Shenyang University of Technology, Shenyang, 110023, China

    Gang Zhang

  2. School of Information Science and Engineering, Shenyang University of Technology, Shenyang, 110023, China

    Gang Zhang, Yuding Ke & Weikang Zhang

  3. Faculty of Computers and Information, Computer Science Department, South Valley University, Luxor, Egypt

    M. Hassaballah

Authors
  1. Gang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuding Ke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Weikang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. M. Hassaballah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gang Zhang .

Editor information

Editors and Affiliations

  1. Department of Computer Science, South Valley University, Luxor, Egypt

    Mahmoud Hassaballah

  2. Department of Information Technology, Zagazig University, Zagazig, Egypt

    Khalid M. Hosny

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Zhang, G., Ke, Y., Zhang, W., Hassaballah, M. (2019). Advances and Trends in Video Face Alignment. In: Hassaballah, M., Hosny, K. (eds) Recent Advances in Computer Vision. Studies in Computational Intelligence, vol 804. Springer, Cham. https://doi.org/10.1007/978-3-030-03000-1_3

Download citation

Publish with us

Policies and ethics

Search

Navigation