Skip to main content
SpringerLink
Log in

Face Recognition Through Different Facial Expressions

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Face recognition has become an accessible issue for experts as well as ordinary people as it is a focal non-interfering biometric modality. In this paper, we introduced a new approach to perform face recognition under varying facial expressions. The proposed approach consists of two main steps: facial expression recognition and face recognition. They are two complementary steps to improve face recognition across facial expression variation. In the first step, we selected the most expressive regions responsible for facial expression appearance using the Mutual Information technique. Such a process helps not only improve the facial expression classification accuracy but also reduce the features vector size. In the second step, we used the Principal Component Analysis (PCA) to build EigenFaces for each facial expression class. Then, a face recognition is performed by projecting the face onto the corresponding facial expression Eigenfaces. The PCA technique significantly reduces the dimensionality of the original space since the face recognition is carried out in the reduced Eigenfaces space. An experimental study was conducted to evaluate the performance of the proposed approach in terms of face recognition accuracy and spatial-temporal complexity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8

Similar content being viewed by others

References

  1. Neurological and Psychological Mechanisms for Producing Facial Expressions. Psychological Bulletin (American Psychological Association Inc.)(1995).

  2. Abbas, A. (2010). Face identification using multiwavelet-based neural network. Ph.D. thesis, University of Baghdad.

  3. Ahonen, T., & H.A.P.M. (2004). Face recognition with local binary patterns, the European Conference on Computer Vision.

  4. Arandjelovic, O., & S.G.F.J.C.R.D.T. (2005). Face recognition with image sets using manifold density divergence. In: CVPR.

  5. Bartlett, M.S., & L.G.F.I.M.R. (2003). Real time face detection and facial expression recognition: Development and application to human computer interaction. In: CVPR Workshop on CVPR for HCI.

  6. Barton, J.J., & Z.J.K.J.P. (2003). Perception of global facial geometry in the inversion effect and prosopagnosia. Neuropsychologia.

  7. Belhumeur P.N., & H.J.K.D. (1997). Eigenfaces vs. fisherfaces, Recognition using class specific linear projection. J. IEEE Trans (PAMI).

  8. Bledsoe, W.W. (1964). The model method in facial recognition. In: Panoramic research Inc.

  9. Bookstein, F. (1989). Principal warps: Thin-plate splines and the decomposition of de-formations. IEEE Transaction Pattern Analog Machine Intelligence.

  10. Boughrara, H., & C.L.B.A.C.C.M. (2012). Face recognition under varying facial expression based on perceived facial images and local feature matching. International Conference on Information Technology and e-Services.

  11. Braje, W.L. (2003). Illumination encoding in face recognition: Effect of position shift. Journal of Vision.

  12. Bruce, V. (1988). Recognizing faces: Lawrence Erlbaum Associates.

  13. Bruce, V., & Y.A. (1998). In the eye of the beholder: The science of face perception. In: Oxford University Press.

  14. Bruner, I. S., & T.R. (1954). The perception of people. In Handbook of Social Psychology.

  15. Cevikalp, H., & T.B. (2010). Face recognition based on image sets. In: CVPR.

  16. Cooper, E.E., & W.T.J. (2000). Differences in the coding of spatial relations in face identification and basic-level object recognition. Journal of Experimental Psy-chology, Learning, Memory, and Cognition.

  17. Cootes, T., & E.G.T.C. Active appearance models. In: Computer Vision ECCV98 1998.

  18. Cover, T.M., & H.P. (1967). Nearest neighbor pattern classification. IEEE Transactions on Information Theory.

  19. Dreuw, P., & S.P.H.H.N.H.A.G. (2009). Surf-face: Face recognition under viewpoint consistency constraints. In: BMVC.

  20. Duchenne, G. Mecanisme de la Physionomie Humaine (1862).

  21. Ekman, P., & F.W. (1971). Constants across cultures in the face and emotion. Journal of Personality Social Psychology.

  22. Ellis, H. D., & S.J.W.D.G.M. (1979). Identification of familiar and unfamiliar faces from internal and external features: Some implications for theories of face recognition. Perception.

  23. Fan, W., & Y.D. (2006). Locally linear models on face appearance manifolds with application to dual-subspace based classification. In: CVPR.

  24. Florent, B., & P.S. (2007). Agents Emotionnels: Etat de lart. UPMC.

  25. Freitas-Magalhes, A. (2007). The Psychology of Emotions: The Allure of Human Face. Oporto: University Fernando Pessoa Press.

  26. Freund, Y., & S.R.E. (1997). A decision-theoretic generalization of on-line learning and an application to boosting. Journal of Computer and System Sciences.

  27. Gheorghita, G., Rajkumar, K., S.K. (2014). Gradient orientation-based pca subspace for novel face recognition. Journal.

  28. Goldstein, A. J., & H.L.D.L.A.B. (1971). Identification of human faces. In: Proceedings of the IEEE.

  29. Gross, R., & M.I.B.S (2004). Appearance-based face recognition and light- fields. IEEE Transaction Pattern Analysis Machine Intelligence.

  30. Hamm, J. (2008). L.D.D.: Grassmann discriminant analysis: A unifying view on subspace-based learning. In: Int. Conf. Mach. Learn.

  31. HU, Y.Q., & M.A.S.O.R. (2012). Face recognition using sparse approximated nearest points between image sets. IEEE PAMI.

  32. Mansouri, H.K., & M. T. M. (2013). Hierarchical structure based convolutional neural network for face recognition. International Journal of Computational Intelligence and Applications.

  33. Kanade, T., & C. J. T. Y. (2000). Comprehensive database for facial expression analysis. International Conference on Automatic Face and Gesture Recognition.

  34. Kanade, T. (1973). Picture processing system by computer complex and recognition of human faces. Ph.D. thesis. Kyoto University.

  35. Kaya, Y., & K. K. (1972). A basic study on human face recognition. In S. Wanatabe (Ed.), Frontiers of pattern recognition.

  36. Keltner, D., & E. P. Facial Expression of Emotion. In Lewis, M., & Haviland-Jones, J. (Eds.), Handbook of emotions, 2nd edn. New York: Guilford Publications.

  37. Khan, M. A., & C. S. X. H. A. (2014). titre. In: On the application of AAM-based systems in face recognition, EUSIPCO.

  38. Khan, M. A., & X. C. A. H. (2013). Multi-model aam framework for face image modeling. In: International Conference on Digital Signal Processing (DSP).

  39. Kim, T.-K., & A. O. C. R. (2007). Boosted manifold principal angles for image set-based recognition. Pattern Recognition.

  40. Kim, T.-K., & K. J. C. R. (2006). Incremental learning of locally orthogonal subspaces for set-based object recogntion. In: BMVC.

  41. Lee, K.-C., & H.J.Y.M.H.K.D. (2003). Video-base face recognition using probabilistic appearance manifolds. In: CVPR.

  42. Lenc, L., & K. P. (2014). Automatically detected feature positions for lbp based face recognition. In: AIAI.

  43. Li, G., & Z. J. W. Y. F. W. (2006). Face recognition using a neural network simulating olfactory systems. In Wang, J., Yi, Z., urada, J.M., Lu, B.-L., Yin, H. (Eds.)

  44. Littlewort, G., & B. M. F. I. S. J. M. J. (2004). Dynamics of facial expression extracted automatically from video. In: IEEE Workshop on Face Pro-cessing in Video.

  45. Liu, C.H., & C. A. (2002). Reassessing the view effect in face recognition. Cognition.

  46. Liu, X., & C. T. (2003). Video-based face recognition using adaptive hidden markov models. In: CVPR.

  47. Liu, W., & L. Z. T. X. (2006). Spatio-temporal embedding for statistical face recognition from video. In: ECCV.

  48. Lowe, D. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision.

  49. Lu, K., & H. X. Z. J. (2006). Semi-supervised support vector learning for face recognition. In Wang, J., Yi, Z., urada, J.M., Lu, B.-L., Yin, H. (Eds.)

  50. Lyons, M., & B. J. A. S. (1991). Automatic classification of single facial images. IEEE Transaction Pattern Analysis Machine Intelligence.

  51. Metaxas, D. N., & J. H. B. L. P. Y. Q. L. L. Z. (2012). Learning active facial patches for expression analysis. IEEE Conference on Computer Vision and Pattern Recognition.

  52. Mliki, H., & H.M. (2014). Discriminative regions selection for facial expression recognition. IJCSI International Journal of Computer Science Issues.

  53. Mliki, H., & H. M. B. A. H. (2013). Mutual information-based facial ex-pression recognition. Sixth International Conference on Machine Vision.

  54. Moon, H., & P. P. J. (2001). Computational and performance aspects of pca-based face-recognition algorithms. Perception.

  55. Nishiyama, M., & Y. M. S. T. W. T. K. T. Y. O. (2007). Recognizing faces of moving people by hierarchical image-set matching. In: CVPR.

  56. Ojala, T., & P. M. H. D. (1996). A comparative study of texture measures with classification based on feature distributions. Pattern Recognition.

  57. Pevenec, P., & A. J. (1996). Local feature analysis : a general statistical theory for object representation. Network, Computation in Neural Systems.

  58. Rubiolo, M., & S. G. M. D. (2013). Compressing arrays of classifiers using volterra-neural network: application to face recognition. Neural Computing and Applications.

  59. Sadr, J., & J. I. S. P. (2003). The role of eyebrows in face recognition. Perception.

  60. Shakhnnarvovich, G., & F. J. W. D. T. (2002). Face recognition from long-term observation. In: ECCV.

  61. Shan, C., & G.S.M.P. (2005). Robust facial expression recognition using local binary patterns. In: IEEE International Conference on Image Processing (ICIP).

  62. Shan, C., & G.S.M.P. (2009). Facial expression recognition based on local binary patterns: A comprehensive study. In: Image Vis. Comput.

  63. Shih F. Y., & C. C. (2004). Automatic extraction of head and face boundaries and facial features. Inf. Sci.

  64. Smaoui, S., & H. M. (2009). Face recognition based on facial feature training. In: ACS/IEEE International Conference on Computer Systems and Applications.

  65. Sobottka, K., & P. I. (1997). A fully automatic approach to facial feature detection and tracking. Audio- and Video-Based Biometric Person Authentication.

  66. Stallkamp, J., & E. H. K. S. R. (2007). Video-based face recognition on real-world data. In: ICCV.

  67. Stanimir, S., & S. S. (2014). Modeling the process of face recognition with pattern neural network using a generalized net. In: IEEE Conference on Intelligent Systems.

  68. Turk, M., & P. A. (1991). Face recognition using eigenfaces. In: IEEE Conference on Computer Vision and Pattern Recognition.

  69. Valentine, T. (1991). A unified account of the effects of distinctiveness, inversion, and race in face recognition. Quarterly Journal of Experimental Psychology.

  70. Vapnik, V. (1998). Statistical Learning Theory: Wiley.

  71. Viola, P. (2001). J.M.: Robust real-time object detection. In: IJCV.

  72. Wang, R., & C. X. (2009). Manifold discriminant analysis. In: CVPR.

  73. Wang, T., & S. P. (2009). Kernel grassmannian distances and discriminant analysis for face recognition from image sets. Pattern Recognition.

  74. Wang, T., & S. P. (2009). Kernel grassmannian distances and discriminant analysis for face recogntion from image sets. PRL.

  75. Wang, R., & S. S. C. X. G. W. (2008). Manifold-manifold distance with application to face recogniton based on image set. In: CVPR.

  76. Wiskott, L., & F. J. M. d. M. C. V. (1997). Face recognition by elastic bunch graph matching. IEEE Trans. Patt. Anal. Mach Intelli.

  77. Wright, J., & Y. A. G. A. S. S. M. Y. (2009). Robust face recognition via sparse representation. Pattern Anal. Mach Intell.

  78. Xiaoming, Z., & S. Z. (2011). Facial expression recognition based on local binary patterns, and kernel discriminant isomap. Sensors.

  79. Xie, X., & L. K. (2008). Face recognition using elastic local reconstruction based on a single face image. In: Pattern Recognition 41.

  80. Xue, Z., & L. S. T. E. (2003). Bayesian shape model for facial feature extraction and recognition. Patt. Recogn.

  81. Yang, M., & Z. P. G. L. V. Z. L. (2013). Face recognition based on regularized nearest points between image sets. In: IEEE Conference. Face Recognit.

  82. Zhang, X., & G. Y. L. M. (2005). Multilevel quadratic variation minimization for 3d face modeling and virtual view synthesis. In: the International Multimedia Modelling Conference.

  83. Zhang, X., & G. Y. L. M. (2008). Recognizing rotated faces from frontal and side views: an approach towards effective use of mugshot databases. IEEE Trans. Inf. Forensic Secur.

  84. Zhao, W., & K. A. C. R. S. D. W. J. (1998). Discriminant analysis of principal components for face recognition. In Face Recognitio, From Therory to Applications.

  85. Zhou, S., & C. R. (2002). Probabilistic human recognition from video. In: ECCV.

  86. Zhou, X., & J. W. T. J. (2014). Enhanced sample selection for svm on face recognition. In APWeb Workshophs.

  87. Zhou, W., & P. X. Z. Z. (2006). Parts-based holistic face recognition with rbf neural networks In Wang, J., Yi, Z., urada, J.M., Lu, B.-L., Yin, H. (Eds.)

  88. Zobel, M., & G. A. P. D. D. J. N. H. (2000). Robust facial feature localization by coupled features. In 4th IEEE International Conference on Automatic Face and Gesture Recognition.

Download references

Author information

Authors and Affiliations

  1. MIRACL-FSEG, University of Sfax, 3018, Sfax, Tunisia

    Hazar Mliki

  2. MIRACL-FS, University of Sfax, 3018, Sfax, Tunisia

    Emna Fendri & Mohamed Hammami

Authors
  1. Hazar Mliki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Emna Fendri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mohamed Hammami
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hazar Mliki.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mliki, H., Fendri, E. & Hammami, M. Face Recognition Through Different Facial Expressions. J Sign Process Syst 81, 433–446 (2015). https://doi.org/10.1007/s11265-014-0967-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-014-0967-z

Keywords

Search

Navigation