Skip to main content
SpringerLink
Log in

On the use of dynamic features in face biometrics: recent advances and challenges

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The way a person is moving his/her head and facial parts (such as the movements of the mouth when a person is talking) defines so called facial dynamics and characterizes personal behaviors. An emerging direction in automatic face analysis consists of also using such dynamic cues, in addition to facial structure, in order to enhance the performance of static image-based methods. This is inspired by psychophysical and neural studies indicating that behavioral characteristics do also provide valuable information to face analysis in the human visual system. This survey article presents the motivations, reviews the recent developments and discusses several other important issues related to the use of facial dynamics in computer vision. As a case study of using facial dynamics, two LBP-based baseline methods are considered and experimental results in different face-related problems, including face recognition, gender recognition, age estimation and ethnicity classification are reported and discussed. Furthermore, remaining challenges are highlighted and some promising directions are pointed out.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. http://www.biosecure.info/

  2. Ahonen T., Hadid A., Pietikäinen M.: Face description with local binary patterns: application to face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 28(12), 2037–2041 (2006)

    Article  Google Scholar 

  3. Baluja S., Rowley H.: Boosting sex identification performance. Int. J. Comput. Vis. 71, 111–119 (2007)

    Article  Google Scholar 

  4. Bassili J.: Emotion recognition: the role of facial movement and the relative importance of upper and lower areas of the face. J. Pers. Soc. Psychol. 37, 2049–2059 (1979)

    Article  Google Scholar 

  5. Bowyer, K., Chang, K., Flynn, P.: A survey of approaches to three-dimensional face recognition. In: International Conference on Automatic Face and Gesture Recognition, vol. 1, pp. 358–361 (2004)

  6. Chellappa, R., Wilson, C.L., Sirohey, S.: Human and machine recognition of faces: a survey. In: IEEE, vol. 83, no. 5, pp. 705–740 (1995)

  7. Chen, L.F., Liao, H.-Y.M., Lin, J.C.: Person identification using facial motion. In: International Conference on Image Processing, 2001, vol. 2, pp. 677–680 (2001)

  8. CRIM: http://www.crim.ca/

  9. Doucet A., Godsill S., Andrieu C.: On sequential monte carlo sampling methods for bayesian filtering. Stat. Comput. 10(3), 197–208 (2000)

    Article  Google Scholar 

  10. Fasel B., Luettin J.: Automatic facial expression analysis: a survey. Pattern Recognit. 36, 259–275 (2003)

    Article  MATH  Google Scholar 

  11. Foucher, S., Lalibert, T.F., Boulianne, G., Gagnon, L.: A dempster-shafer based fusion approach for audio-visual speech recognition with application to large vocabulary french speech. In: IEEE International Conference on Acoustics, Speech and Signal Processing, 2006 (ICASSP 2006), pp. I597–I600 (2006)

  12. Freund Y., Schapire R.: A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1), 119–139 (1997)

    Article  MathSciNet  MATH  Google Scholar 

  13. Geng X., Zhou Z.H., Smith-Miles K.: Automatic age estimation based on facial aging patterns. IEEE Trans. Pattern Anal. Mach. Intell. 29(12), 2234–2240 (2007)

    Article  Google Scholar 

  14. Gross, R., Shi, J.: The CMU motion of body (mobo) database. Technical report (2001)

  15. Hadid, A., Pietikäinen, M.: Combining motion and appearance for gender classification from video sequences. In: 19th International Conference on Pattern Recognition (ICPR 2008), p. 4 (2008)

  16. Hadid A., Pietikäinen M.: Combining appearance and motion for face and gender recognition from videos. Pattern Recognit. 42(11), 2818–2827 (2009)

    Article  Google Scholar 

  17. Hadid, A., Pietikäinen, M., Li, S.Z.: Learning personal specific facial dynamics for face recognition from videos. In: IEEE International Workshop on Analysis and Modeling of Faces and Gestures (in conjunction with ICCV 2007), pp. 1–15 (2007)

  18. Heisele B., Ho P., Wu J., Poggio T.: Face recognition: component based versus global approaches. Comput. Vis. Image Underst. 91(1–2), 6–21 (2003)

    Article  Google Scholar 

  19. Hill H., Johnston A.: Categorizing sex and identity from the biological motion of faces. Curr. Biol. 11(11), 880–885 (2001)

    Article  Google Scholar 

  20. Hjelmas E., Low B.K.: Face detection: a survey. Comput. Vis. Image Underst. 83(3), 236–274 (2001)

    Article  MATH  Google Scholar 

  21. Huang, K.S., Trivedi, M.M.: Streaming face recognition using multicamera video arrays. International Conference on Pattern Recognition, vol. 4, p. 40213 (2002)

  22. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 46–53 (2000)

  23. Kittler, J., Nixon, M.S. (eds.): Audio- and video-based biometrie person authentication. Lecture Notes in Computer Science, vol. 2688. Springer (2003)

  24. Knight B., Johnston A.: The role of movement in face recognition. Vis. Cognit. 4, 265–274 (1997)

    Article  Google Scholar 

  25. Koreman, J., Morris, A., Jassim, S., Sellahewa, H., Chollet, G., Aversano, G., Salicetti, S., Allano, L. (eds.): Multi-modal biometric authentication on the Secure Phone PDA (2006)

  26. Lanitis A., Taylor C., Cootes T.: Towards automatic simulation of aging affects on face images. IEEE Trans. Pattern Anal. Mach. Intell. 24, 442–455 (2002)

    Article  Google Scholar 

  27. Lee, B., Hasegawa-Johnson, M., Goudeseune, C., Kamdar, S., Borys, S., Liu, M., Huang, T.: Avicar: Audio-visual speech corpus in a car environment. In: Proceedings of International Conference on Spoken Language (2004)

  28. Lee K.C., Ho J., Yang M.H., Kriegman D.: Visual tracking and recognition using probabilistic appearance manifolds. Comput. Vis. Image Underst. 99(3), 303–331 (2005)

    Article  Google Scholar 

  29. Li B., Chellappa R.: Face verification through tracking facial features. J. Opt. Soc. Am. 18, 2969–2981 (2001)

    Article  Google Scholar 

  30. Li B., Chellappa R.: A generic approach to simultaneous tracking and verification in video. IEEE Trans. Image Process. 11(5), 530–544 (2002)

    Article  Google Scholar 

  31. Li, S.Z., Jain, A.K. (eds): Handbook of Face Recognition. Springer, New York (2005)

    MATH  Google Scholar 

  32. Li, Y.: Dynamic face models: construction and applications. Ph.D. thesis, Queen Mary, University of London (2001)

  33. Liu J.S., Chen R.: Sequential monte carlo methods for dynamic systems. J. Am. Stat. Assoc. 93, 1032–1044 (1998)

    Article  MATH  Google Scholar 

  34. Liu, X., Chen, T.: Video-based face recognition using adaptive hidden markov models. In: IEEE International Conference on Computer Vision and Pattern Recognition, pp. 340–345 (2003)

  35. Matta, F., Dugelay, J.L.: Person recognition using human head motion information. In: Articulated Motion and Deformable Objects, pp. 326–335 (2006)

  36. Matta, F., Dugelay, J.L.: Video face recognition: a physiological and behavioural multimodal approach. In: ICIP 2007, 14th IEEE International Conference on Image Processing, 16–19 Sept 2007, San Antonio, USA (2007)

  37. Matta F., Dugelay J.L.: Person recognition using facial video information: a state of the art. Image Vis. Comput. 20(3), 180–187 (2009)

    Google Scholar 

  38. Matta, F., Saeed, U., Mallauran, C., Dugelay, J.L.: Facial gender recognition using multiple sources of visual information. In: MMSP 2008, 10th IEEE International Workshop on MultiMedia Signal Processing, 8–10 Oct 2008, Cairns, Queensland, Australia (2008)

  39. Meng, H., Ching, P.C., Lee, T., Mak, M.W., Mak, B., Moon, Y.S., Siu, M.H., Tang, X., Hui, H., Lee, A., Lo, W.K., Ma, B. (eds.): The Multi-Biometric, Multi-Device and MultiLingual (M3) Corpus (2006)

  40. Messer, K., Matas, J., Kittler, J., Lüttin, J., Maitre, G.: XM2VTSDB: The extended M2VTS database. In: Audio- and Video-Based Biometric Person Authentication, AVBPA’99, March 1999, 16 IDIAP–RR 99-02, pp. 72–77. Washington, DC (1999)

  41. Moghaddam B.: Principal manifolds and probabilistic subspaces for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. 24, 780–788 (2002)

    Article  Google Scholar 

  42. Moghaddam B., Yang M.H.: Learning gender with support faces. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 707–711 (2002)

    Article  Google Scholar 

  43. Murase H., Nayar S.K.: Visual learning and recognition of 3-d objects from appearance. Int. J. Comput. Vis. 14(1), 5–24 (1995)

    Article  Google Scholar 

  44. Ojala T., Pietikäinen M., Harwood D.: A comparative study of texture measures with classification based on feature distributions. Pattern Recognit. 29, 51–59 (1996)

    Article  Google Scholar 

  45. Ojala T., Pietikäinen M., Mäenpää T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Trans. Pattern Anal. Mach. Intell. 24, 971–987 (2002)

    Article  Google Scholar 

  46. O’Toole A.J., Roark D.A., Abdi H.: Recognizing moving faces: a psychological and neural synthesis. Trends Cognit. Sci. 6, 261–266 (2002)

    Article  Google Scholar 

  47. Pantic M., Rothkrantz L.: Automatic analysis of facial expressions: the state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1455 (2000)

    Article  Google Scholar 

  48. Pigeon, S., Vandendorpe, L.: The M2VTS multimodal face database (release 1.00). In: AVBPA’97: Proceedings of the First International Conference on Audio- and Video-Based Biometric Person Authentication, pp. 403–409. Springer, London, UK (1997)

  49. Popovici, V., Thiran, J., Bailly-Bailliere, E., Bengio, S., Bimbot, F., Hamouz, M., Kittler, J., Mariethoz, J., Matas, J., Messer, K., Ruiz, B., Poiree, F.: The BANCA database and evaluation protocol. In: 4th International Conference on Audio- and Video-Based Biometric Person Authentication, Guildford, UK. Lecture Notes in Computer Science, vol. 2688, pp. 625–638. SPIE (2003)

  50. Saeed, U., Matta, F., Dugelay, J.L.: Person recognition based on head and mouth dynamics. In: MMSP 2006, IEEE International Workshop on Multimedia Signal Processing, 3–6 Oct 2006, Victoria, Canada (2006)

  51. Samal A., Iyengar P.: Automatic recognition and analysis of human faces and facial expression: a survey. Pattern Recognit. 25(1), 65–77 (1992)

    Article  Google Scholar 

  52. Sanderson C., Paliwal K.K.: Noise compensation in a person verification system using face and multiple speech feature. Pattern Recognit. 36(2), 293–302 (2003)

    Article  Google Scholar 

  53. Shan, C., Gong, S., McOwan, P.: Learning gender from human gaits and faces. In: IEEE International Conference on Advanced Video and Signal based Surveillance, pp. 505–510 (2007)

  54. Tistarelli M., Bicego M., Grosso E.: Dynamic face recognition: From human to machine vision. Image Vis. Comput. 27(3), 222–232 (2009)

    Article  Google Scholar 

  55. Yang M.H., Kriegman D.J., Ahuja N.: Detecting faces in images: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 24, 34–58 (2002)

    Article  Google Scholar 

  56. Zhao G., Pietikäinen M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)

    Article  Google Scholar 

  57. Zhao W., Chellappa R., Phillips P.J., Rosenfeld A.: Face recognition: a literature survey. ACM Comput. Surv. 34(4), 399–458 (2003)

    Article  Google Scholar 

  58. Zhou, S., Chellappa, R.: Probabilistic human recognition from video. In: European Conference on Computer Vision, pp. 681–697 (2002)

  59. Zhou S., Chellappa R., Moghaddam B.: Visual tracking and recognition using appearance-adaptive models in particle filters. IEEE Trans. Image Process. 13, 1434–1456 (2004)

    Google Scholar 

  60. Zhou, S., Krueger, V., Chellappa, R.: Face recognition from video: a condensation approach. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 221–228 (2002)

  61. Zhou S., Kruger V., Chellappa R.: Probabilistic recognition of human faces from video. Comput. Vis. Image Underst. 91, 214–245 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Machine Vision Group, University of Oulu, Oulu, Finland

    Abdenour Hadid & Matti Pietikäinen

  2. EURECOM, Sophia Antipolis, France

    Jean-Luc Dugelay

Authors
  1. Abdenour Hadid
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jean-Luc Dugelay
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Matti Pietikäinen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abdenour Hadid.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hadid, A., Dugelay, JL. & Pietikäinen, M. On the use of dynamic features in face biometrics: recent advances and challenges. SIViP 5, 495–506 (2011). https://doi.org/10.1007/s11760-011-0247-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-011-0247-3

Keywords

Search

Navigation