Skip to main content
Springer Nature Link
Log in

Multimodal Facial Expression Recognition Based on 3D Face Reconstruction from 2D Images

  • Conference paper
  • First Online:
Face and Facial Expression Recognition from Real World Videos (FFER 2014)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 8912))

  • 1220 Accesses

Abstract

In this paper, a novel feature extraction method was proposed for facial expression recognition. A 3D Facial Expression Generic Elastic Model (3D FE-GEM) was proposed to reconstruct an expression-invariant 3D model of each human face in the present database using only a single 2D frontal image with/without facial expressions. Then, the texture and depth of the face were extracted from the reconstructed model. Afterwards, the Gabor filter bank was applied to both texture and reconstructed depth of the face to extract the feature vectors from both texture and reconstructed depth images. Finally, by combining 2D and 3D feature vectors, the final feature vectors are generated and classified by the Support Vector Machine (SVM). Favorable outcomes were acquired for facial expression recognition on the available image database based on the proposed method compared to several state-of-the-arts in facial expression recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

Similar content being viewed by others

References

  1. Li, S.Z., Jain, A.K., Tian, Y.L., Kanade, T., Cohn, J.F.: Facial expression analysis. In: Handbook of Face Recognition, pp. 247–275. Springer, New York (2005). 10.1007/0-387-27257-7-12

  2. Bourbakis, N., Kakumanu, P.: Skin-based face detection-extraction and recognition of facial expressions. In: Bunke, H., Kandel, A., Last, M. (eds.) Applied Pattern Recognition. SCI, vol. 91, pp. 3–27. Springer, Heidelberg (2008)

    Google Scholar 

  3. Cheddad, A., Mohamad, D., Manaf, A.A.: Exploiting voronoi diagram properties in face segmentation and feature extraction. Pattern Recognition 41(12), 3842–3859 (2008)

    Article  MATH  Google Scholar 

  4. Xie, X., Lam, K.-M.: Facial expression recognition based on shape and texture. Pattern Recognition 42(5), 1003–1011 (2009)

    Article  Google Scholar 

  5. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing 27(6), 803–816 (2009)

    Article  Google Scholar 

  6. Tian, Y.L.: Evaluation of face resolution for expression analysis. In: Conference on Computer Vision and Pattern Recognition Workshop, CVPRW 2004, p. 82, June 2004

    Google Scholar 

  7. Pantic, M., Rothkrantz, L.J.M.: Automatic analysis of facial expressions: The state of the art. IEEE Trans. Pattern Anal. Mach. Intell. 22(12), 1424–1445 (2000)

    Article  Google Scholar 

  8. Tian, Y.: Evaluation of face resolution for expression analysis. In: CVPR Workshop on Face Processing in Video (2004)

    Google Scholar 

  9. Shan, C., Gong, S., McOwan, P.W.: Facial expression recognition based on Local Binary Patterns: A comprehensive study. Image and Vision Computing 27, 803–816 (2009)

    Article  Google Scholar 

  10. Moore, S., Bowden, R.: Local binary patterns for multi-view facial expression recognition. Computer Vision and Image Understanding 115, 541–558 (2011)

    Article  Google Scholar 

  11. Ojansivu, V., Heikkilä, J.: Blur insensitive texture classification using local phase quantization. In: Elmoataz, A., Lezoray, O., Nouboud, F., Mammass, D. (eds.) ICISP 2008. LNCS, vol. 5099, pp. 236–243. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  12. Turk, M., Pentland, A.: Eigenfaces for Recognition. Journal of Cognitive Neuroscience 3(1), 71–86 (1991)

    Article  Google Scholar 

  13. Etemad, K., Chellappa, R.: Discriminant analysis for recognition of human face images. Journal of Optical Society of America A 14, 1724–1733 (1997)

    Article  Google Scholar 

  14. Heo, J., Savvides, M.: 3-D generic elastic models for fast and texture preserving 2-D novel pose synthesis. IEEE Transactions on Information Forensics and Security 7, 563–576 (2012)

    Article  Google Scholar 

  15. Zhao, X., Dellandréa, E., Zou, J., Chen, L.: A unified probabilistic framework for automatic 3D facial expression analysis based on a Bayesian belief inference and statistical feature models. Image and Vis. Comp. 31, 231–245 (2013)

    Article  Google Scholar 

  16. Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BIOID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  17. Liu, C., Wechsler, H.: Gabor feature classifier for face recognition. In: Proceedings of the ICCV, pp. 270–275 (2001)

    Google Scholar 

  18. Hsu, C.-W., Chang, C.-C., Lin, C.-J.: A Practical Guide to Support Vector Classification. Tech. Rep., Taipei (2003)

    Google Scholar 

  19. Langner, O., Dotsch, R., Bijlstra, G., Wigboldus, D.H.J.: Presentation and validation of the Radboud faces database. Cognition & Emotion 24 (2010)

    Google Scholar 

  20. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust Face Recognition via Sparse Representation. IEEE Transactions on Pattern Analysis and Machine Intelligence 31, 210–227 (2009)

    Article  Google Scholar 

  21. Saragih, J., Lucey, S., Cohn, J.: Deformable Model Fitting by Regularized Landmark Mean-Shift. International Journal of Computer Vision 91, 200–215 (2011)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Amirkabir University of Technology, Tehran, Iran

    Ali Moeini

  2. Semnan University, Semnan, Iran

    Hossein Moeini

Authors
  1. Ali Moeini
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hossein Moeini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ali Moeini .

Editor information

Editors and Affiliations

  1. Computer and Systems Engineering, Pensselaer Polytechnic Institute, Troy, New York, USA

    Qiang Ji

  2. Department of Architecture, Design, and Media Technology, Aalborg, Denmark

    Thomas B. Moeslund

  3. Department of Computer Science, Stevens Institute of Technology, Hoboken, New Jersey, USA

    Gang Hua

  4. Department of Architecture, Design, and Media Technology, Aalborg, Denmark

    Kamal Nasrollahi

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Moeini, A., Moeini, H. (2015). Multimodal Facial Expression Recognition Based on 3D Face Reconstruction from 2D Images. In: Ji, Q., B. Moeslund, T., Hua, G., Nasrollahi, K. (eds) Face and Facial Expression Recognition from Real World Videos. FFER 2014. Lecture Notes in Computer Science(), vol 8912. Springer, Cham. https://doi.org/10.1007/978-3-319-13737-7_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-13737-7_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-13736-0

  • Online ISBN: 978-3-319-13737-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics