Skip to main content
SpringerLink
Log in

Incorporating prior knowledge from the new person into recognition of facial expression

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Inter-subject variability plays an important role in the performance of facial expression recognition. Therefore, several methods have been developed to bring the performance of a person-independent system closer to that of a person-dependent one. These techniques need different samples from a new person to increase the generalization ability. We have proposed a new approach to address this problem. It employs the person’s neutral samples as prior knowledge and a synthesis method based on the subspace learning to generate virtual expression samples. These samples have been incorporated in learning process to learn the style of the new person. We have also enriched the training data set by virtual samples created for each person in this set. Compared with previous studies, the results showed that our approach can perform the task of facial expression recognition effectively with better robustness for corrupted data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cohen, I., Sebe, N., Garg, A., et al.: Facial expression recognition from video sequences: temporal and static modeling. Comput. Vis. Image Underst. 91, 160–187 (2003)

    Article  Google Scholar 

  2. Maronidis, A., Bolis, D., Tefas, A., et al.: Improving subspace learning for facial expression recognition using person dependent and geometrically enriched training sets. Neural Netw. 24(8), 814–823 (2011)

    Article  Google Scholar 

  3. Zhang, X.-Y., Liu, C.-L.: Style transfer matrix learning for writer adaptation. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 393–400 (2011)

  4. Xuan, T.W.: Rapid Speaker Adaptation by Variable Reference Model Subspace. Université De Rennes, Rennes (2008)

    Google Scholar 

  5. Niyogi, P., Girosi, F., Poggio, T.: Incorporating prior information in machine learning by creating virtual examples. Proc. IEEE 86(11), 2196–2209 (1998)

    Article  Google Scholar 

  6. Jabid, M.H.K.T., Chae, O.: Robust facial expression recognition based on local directional pattern. ETRI J. 32(5), 784–794 (2010)

    Article  Google Scholar 

  7. Cavalin, P.R., Sabourin, R., Suen, C.Y., et al.: Evaluation of incremental learning algorithms for HMM in the recognition of alphanumeric characters. Pattern Recognit. 42(12), 3241–3253 (2009)

    Article  MATH  Google Scholar 

  8. Leggetter, C.J., Woodland, P.C.: Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models. Comput. Speech Lang. 9(2), 171–185 (1995)

    Article  Google Scholar 

  9. Kuhn, R., Junqua, J.-C., Nguyen, P., et al.: Rapid speaker adaptation in eigenvoice space. IEEE Trans. Speech Audio Process. 8(6), 695–707 (2000)

    Article  Google Scholar 

  10. Kienzle, W., Chellapilla, K.: Personalized handwriting recognition via biased regularization. In: Proceedings of the 23rd International Conference on Machine learning, Pittsburgh, Pennsylvania, pp. 457–464 (2006)

  11. Zhang, X.-Y., Liu, C.-L.: Writer adaptation with style transfer mapping. IEEE Trans. Pattern Anal. Mach. Intell. PP(99), 1 (2012)

    Google Scholar 

  12. Patel, N., Zaveri, M.: 3D facial model reconstruction, expressions synthesis and animation using single frontal face image. Signal Image Video Process. 7(5), 889–897 (2013). 2013/09/01

  13. Yu, H., Zhang, J.: Mean value coordinates-based caricature and expression synthesis. Signal Image Video Process. 7(5), 899–910 (2013). 2013/09/01

    Article  Google Scholar 

  14. Sharma, A., Anamika, D.: Facial expression recognition using virtual neutral image synthesis. In: National Conference on Computer Vision Pattern Recognition Image Processing and Graphics, Jaipur, India (2010)

  15. Senechal, T., Bailly, K., Prevost, L.: Impact of action unit detection in automatic emotion recognition. In: Pattern Analysis and Applications, pp. 1–17. Springer, London (2012)

  16. Mohammadzade, H., Hatzinakos, D.: An expression transformation for improving the recognition of expression-variant faces from one sample image per person. In: Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–6 (2010)

  17. Zhao, X., Zhang, S.: Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding. EURASIP J. Adv. Signal Process. 2012(1), 20 (2012)

    Article  Google Scholar 

  18. Shan, C., Gong, S., McOwan, P.: Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  19. Mohammadzade, H., Hatzinakos, D.: Projection into expression subspaces for face recognition from single sample per person. IEEE Trans. Affect. Comput. 4(1), 69–82 (2013)

    Article  Google Scholar 

  20. Bartlett, M.S., Littlewort, G., Frank, M. et al.: Fully automatic facial action recognition in spontaneous behavior. In: 7th International Conference on Automatic Face and Gesture Recognition, 2006. FGR 2006., pp. 223–230 (2006)

  21. Ji, Y., Idrissi, K.: Automatic facial expression recognition based on spatiotemporal descriptors. Pattern Recognit. Lett. 33(10), 1373–1380 (2012)

    Article  Google Scholar 

  22. Dornaika, F., Lazkano, E., Sierra, B.: Improving dynamic facial expression recognition with feature subset selection. Pattern Recognit. Lett. 32(5), 740–748 (2011)

    Article  Google Scholar 

  23. Zhang, Y., Ji, Q.: Active and dynamic information fusion for facial expression understanding from image sequences. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 699–714 (2005)

    Article  Google Scholar 

  24. Schmid, K., Marx, D., Samal, A.: Computation of a face attractiveness index based on neoclassical canons, symmetry, and golden ratios. Pattern Recognit. 41(8), 2710–2717 (2008)

    Article  Google Scholar 

  25. Lucey, P., Cohn, J.F., Kanade, T. et al.: The extended Cohn-Kanade dataset (CK+): a complete dataset for action unit and emotion-specified expression. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 94–101 (2010)

  26. Savran, L.S.A., Bilge, M.T.: Comparative evaluation of 3D vs. 2D modality for automatic detection of facial action units. Pattern Recognit. 45(2), 767–782 (2012)

    Article  Google Scholar 

  27. Gu, W., Ã, C.X., Venkatesh, Y.V., et al.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45(1), 80–91 (2012)

    Article  Google Scholar 

Download references

Acknowledgments

The authors gratefully acknowledge partial funding from the Research Center of Intelligent Signal Processing.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Amirkabir University of Technology, Tehran, Iran

    Amin Mohammadian & Farzad Towhidkhah

  2. Department of Electrical Engineering, Amirkabir University of Technology, Tehran, Iran

    Hassan Aghaeinia

Authors
  1. Amin Mohammadian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hassan Aghaeinia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Farzad Towhidkhah
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Farzad Towhidkhah.

Appendices

Appendix 1: Golden ratios features [24]

Mid-eye distance to inter-ocular distance

Mid-eye distance to nose width

Mouth width to inter-ocular distance

Lips–chin distance to inter-ocular distance

Lips–chin distance to nose width

Inter-ocular distance to eye fissure width

Inter-ocular distance to lip height

Nose width to eye fissure width

Nose width to lip height

Eye fissure width to nose–mouth distance

Lip height to nose–mouth distance

Length of face to width of face

Nose–chin distance to lips–chin distance

Nose width to nose–mouth distance

Mouth width to nose width

Appendix 2: neoclassic rules features [24]

Nose length \(=\) ear length

Inter-ocular distance \(=\) nose width

Inter-ocular distance \(=\) right or left eye fissure width

Mouth width \(= 1.5 \times \) nose width

Face width \(=4 \times \) nose width

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mohammadian, A., Aghaeinia, H. & Towhidkhah, F. Incorporating prior knowledge from the new person into recognition of facial expression. SIViP 10, 235–242 (2016). https://doi.org/10.1007/s11760-014-0732-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-014-0732-6

Keywords

Search

Navigation