Skip to main content
Springer Nature Link
Log in

Personalized Modeling of Facial Action Unit Intensity

  • Conference paper
Advances in Visual Computing (ISVC 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8888))

Included in the following conference series:

  • 2560 Accesses

Abstract

Facial expressions depend greatly on facial morphology and expressiveness of the observed person. Recent studies have shown great improvement of the personalized over non-personalized models in variety of facial expression related tasks, such as face and emotion recognition. However, in the context of facial action unit (AU) intensity estimation, personalized modeling has been scarcely investigated. In this paper, we propose a two-step approach for personalized modeling of facial AU intensity from spontaneously displayed facial expressions. In the first step, we perform facial feature decomposition using the proposed matrix decomposition algorithm that separates the person’s identity from facial expression. These two are then jointly modeled using the framework of Conditional Ordinal Random Fields, resulting in a personalized model for intensity estimation of AUs. Our experimental results show that the proposed personalized model largely outperforms non-personalized models for intensity estimation of AUs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Pantic, M.: Machine analysis of facial behaviour: Naturalistic and dynamic behaviour. Philosophical Transactions of Royal Society B 364, 3505–3513 (2009)

    Article  Google Scholar 

  2. Ekman, P., Friesen, W., Hager, J.: Facial Action Coding System (FACS): Manual. A Human Face (2002)

    Google Scholar 

  3. Mahoor, M., Cadavid, S., Messinger, D., Cohn, J.: A framework for automated measurement of the intensity of non-posed facial action units. In: IEEE CVPR’W, pp. 74–80 (2009)

    Google Scholar 

  4. Vasilescu, M.A.O., Terzopoulos, D.: Multilinear subspace analysis of image ensembles. In: CVPR. vol. 2, p. II-93 (2003)

    Google Scholar 

  5. Mpiperis, I., Malassiotis, S., Strintzis, M.G.: Bilinear models for 3-d face and facial expression recognition. IEEE Trans. on Information Forensics and Security 3, 498–511 (2008)

    Article  Google Scholar 

  6. Valstar, M.F., Pantic, M.: Fully automatic recognition of the temporal phases of facial actions. Systems, Man, and Cybernetics, Part B 42, 28–43 (2012)

    Article  Google Scholar 

  7. Kim, M., Pavlovic, V.: Structured output ordinal regression for dynamic facial emotion intensity prediction. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part III. LNCS, vol. 6313, pp. 649–662. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  8. Hablani, R., Chaudhari, N., Tanwani, S.: Recognition of facial expressions using local binary patterns of important facial parts. Int. Journal of Image Proc. 7, 163 (2013)

    Google Scholar 

  9. Arapakis, I., Athanasakos, K., Jose, J.M.: A comparison of general vs personalised affective models for the prediction of topical relevance. In: Proc. of the 33rd Int. ACM SIGIR Conf. on R&D in Information Retrieval, pp. 371–378 (2010)

    Google Scholar 

  10. Dahmane, M., Meunier, J.: Individual feature–appearance for facial action recognition. In: Kamel, M., Campilho, A. (eds.) ICIAR 2011, Part II. LNCS, vol. 6754, pp. 233–242. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  11. Doulamis, N.: An adaptable emotionally rich pervasive computing system. In: European Signal Processing Conference (EUSIPCO) (2006)

    Google Scholar 

  12. Romera-Paredes, B., Aung, M.S., Pontil, M., Bianchi-Berthouze, N., de C Williams, A., Watson, P.: Transfer learning to account for idiosyncrasy in face and body expressions. In: FG, pp. 1–6 (2013)

    Google Scholar 

  13. Chen, J., Liu, X., Tu, P., Aragones, A.: Learning person-specific models for facial expression and action unit recognition. Pattern Recognition Letters 34, 1964–1970 (2013)

    Article  Google Scholar 

  14. Romera-Paredes, B., Argyriou, A., Berthouze, N., Pontil, M.: Exploiting unrelated tasks in multi-task learning. In: Int. Conf. on Artificial Intelligence and Statistics, pp. 951–959 (2012)

    Google Scholar 

  15. Rudovic, O., Pavlovic, V., Pantic, M.: Context-sensitive Dynamic Ordinal Regression for Intensity Estimation of Facial Action Units. IEEE TPAMI (in press, 2014)

    Google Scholar 

  16. Chu, W.S., Torre, F.D.L., Cohn, J.F.: Selective transfer machine for personalized facial action unit detection. In: CVPR, pp. 3515–3522 (2013)

    Google Scholar 

  17. Reilly, J., Ghent, J., McDonald, J.: Investigating the dynamics of facial expression. In: Bebis, G., et al. (eds.) ISVC 2006. LNCS, vol. 4292, pp. 334–343. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  18. Savrana, A., Sankur, B., Bilgeb, M.: Regression-based intensity estimation of facial action units. In: Image and Vision Computing (2012)

    Google Scholar 

  19. Vasilescu, M.A.O., Terzopoulos, D.: Multilinear image analysis for facial recognition. In: ICPR, vol. 2, p. 20511 (2002)

    Google Scholar 

  20. Wang, H., Ahuja, N.: Facial expression decomposition. In: ICCV, pp. 958–965 (2003)

    Google Scholar 

  21. Lee, C.-S., Elgammal, A.: Facial expression analysis using nonlinear decomposable generative models. In: Zhao, W., Gong, S., Tang, X. (eds.) AMFG 2005. LNCS, vol. 3723, pp. 17–31. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  22. Turk, M., Pentland, A.: Eigenfaces for recognition. Journal of Cognitive Neuroscience 3, 71–86 (1991)

    Article  Google Scholar 

  23. Li, P., Fu, Y., Mohammed, U., Elder, J., Prince, S.: Probabilistic models for inference about identity. TPAMI 34, 144–157 (2012)

    Article  Google Scholar 

  24. Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: ICML, pp. 282–289 (2001)

    Google Scholar 

  25. Winkelmann, R., Boes, S.: Analysis of microdata. Springer (2006)

    Google Scholar 

  26. Mavadati, S., Mahoor, M., Bartlett, K., Trinh, P., Cohn, J.: Disfa: A spontaneous facial action intensity database. IEEE Trans. on Affective Comp. 4(2), 151–160 (2013)

    Article  Google Scholar 

  27. Lucey, P., Cohn, J., Prkachin, K., Solomon, P., Matthews, I.: Painful data: The unbc-mcmaster shoulder pain expression archive database. In: IEEE FG, pp. 57–64 (2011)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Comp. Dept., Imperial College London, UK

    Shuang Yang, Ognjen Rudovic & Maja Pantic

  2. Dept. of Computer Science, Rutgers University, USA

    Vladimir Pavlovic

  3. EEMCS, University of Twente, The Netherlands

    Maja Pantic

Authors
  1. Shuang Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ognjen Rudovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Vladimir Pavlovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Maja Pantic
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University of Nevada at Reno, USA

    George Bebis

  2. NASA Ames Research Center, Moffett Field, CA, USA

    Richard Boyle

  3. Lawrence Berkeley National Laboratory, Berkeley, CA, USA

    Bahram Parvin

  4. Desert Research Institute, Reno, NV, USA

    Darko Koracin

  5. The University of Texas at Dallas, 75080, Richardson, TX, USA

    Ryan McMahan

  6. NextGen Interactions, 27604, Raleigh, NC, USA

    Jason Jerald

  7. Indiana University, 46202, Indianapolis, IN, USA

    Hui Zhang

  8. Microsoft Research, 1 Microsoft Way, 98052, Redmond, WA, USA

    Steven M. Drucker

  9. University of Delaware, 19716-2712, Newark, DE, USA

    Chandra Kambhamettu

  10. Intel Corp., 95054, Sata Clara, CA, USA

    Maha El Choubassi

  11. Computer Graphics and Interactive Media Lab, Department of Computer Science, University of Houston, 77004, Houston, TX, USA

    Zhigang Deng

  12. NVIDIA, 34788, Leesburg, FL, USA

    Mark Carlson

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Yang, S., Rudovic, O., Pavlovic, V., Pantic, M. (2014). Personalized Modeling of Facial Action Unit Intensity. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2014. Lecture Notes in Computer Science, vol 8888. Springer, Cham. https://doi.org/10.1007/978-3-319-14364-4_26

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-14364-4_26

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-14363-7

  • Online ISBN: 978-3-319-14364-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics