Skip to main content

Advertisement

SpringerLink
Log in

Facial expression recognition based on meta probability codes

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

Automatic facial expression recognition has made considerable gains in the body of research available due to its vital role in human–computer interaction. So far, research on this problem or problems alike has proposed a wide verity of techniques and algorithms for both information representation and classification. Very recently, Farajzadeh et al. in Int J Pattern Recognit Artif Intell 25(8):1219–1241, (2011) proposed a novel information representation approach that uses machine-learning techniques to derive a set of new informative and descriptive features from the original features. The new features, so called meta probability codes (MPC), have shown a good performance in a wide range of domains. In this paper, we aim to study the performance of the MPC features for the recognition of facial expression via proposing an MPC-based framework. In the proposed framework any feature extractor and classifier can be incorporated using the meta-feature generation mechanism. In the experimental studies, we use four state-of-the-art information representation techniques; local binary pattern, Gabor-wavelet, Zernike moment and facial fiducial point, as the original feature extractors; and four multiclass classifiers, support vector machine, k-nearest neighbor, radial basis function neural network, and sparse representation-based classifier. The results of the extensive experiments conducted on three facial expression datasets, Cohn–Kanade, JAFFE, and TFEID, show that the MPC features promote the performance of facial expression recognition inherently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. If a given image is changed in terms of scale, position, rotation or a combination of them, its statistical features will remain unchanged.

  2. In the single machine approach, a binary classifier is generalized by adapting its internal operations to a single multiclass optimization problem.

  3. Since the classification rates obtained by SRC using ZM and FFP features are very low, we do not include all of its results in the averaging processes throughout this section in order to avoid unfair comparisons.

  4. The number of fiducial points (34) and their positions in JAFFE dataset, as shown in Fig. 3, are different from those of Cohn–Kanade dataset.

  5. In the averaging process only the results of the systems using LBP features are considered.

References

  1. Farajzadeh N, Pan G, Wu Z, Yao M (2011) Multiclass classification based on meta probability codes. Int J Pattern Recognit Artif Intell 25(8):1219–1241

    Article  MathSciNet  Google Scholar 

  2. Pan G, Xu Y, Wu Z, Li S, Yang LT, Lin M, Liu Z (2011) TaskShadow: toward seamless task migration across smart environments. IEEE Intell Syst 26(3):50–57

    Google Scholar 

  3. Sun J, Wu Z, Pan G (2009) Context-aware smart car: from model to prototype. J Zhejiang Univ Sci A 10(7):1049–1059

    Google Scholar 

  4. Darwin C (1872) The expression of the emotions in man and animals. John Murray, London

  5. Ekman P, Friesen WV (1974) Unmasking the face. Prentice Hall, New Jersey

  6. Pantic M, Rothkrantz LJM (2000) Automatic analysis of facial expressions: the state of the art. IEEE Trans Pattern Anal Mach Intell 22(12):1424–1445

    Article  Google Scholar 

  7. Fasel B, Luettin J (2003) Automatic facial expression analysis: a survey. Pattern Recognit 36(1):259–275

    Article  MATH  Google Scholar 

  8. Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58

    Article  Google Scholar 

  9. Bettadapura V (2012) Face expression recognition and analysis: the state of the art. Technical, College of Computing, Georgia Institute of Technology

  10. Sebe N, Cohen I, Grag A, Huang T (2005) Machine learning in computer vision. Springer, Berlin

  11. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MathSciNet  MATH  Google Scholar 

  12. Ojala T, Pietikinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distribution. Pattern Recognit 29(1):51–59

    Article  Google Scholar 

  13. Lee TS (1996) Image representation using 2D gabor wavelets. IEEE Trans Pattern Anal Mach Intell 18(10):1–13

    Google Scholar 

  14. Teh CH, Chin RT (1988) On image analysis by the methods of moments. IEEE Trans Pattern Anal Mach Intell 10(4):496–513

    Article  MATH  Google Scholar 

  15. Vapnik VN (1998) Statistical learning theory. Wiley, New York

  16. Shakhnarovish G, Darrell T, Indyk P (2005) Nearest-neighbor methods in learning and vision: theory and practice. MIT Press, Boston

  17. Fausett LV (1994) Fundamentals of neural networks: architectures, algorithms, and applications. Prentice-Hall, Upper Saddle River

  18. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  19. Kanade T, Cohn J, Tian Y (2000) Comprehensive database for facial expression analysis. In: Proceedings of the IEEE international conference on automatic face and gesture recognition

  20. Lyons MJ, Budynek J, Akamatsu S (1999) Automatic classification of single facial images. IEEE Trans Pattern Anal Mach Intell 21(12):1357–1362

    Article  Google Scholar 

  21. Chen LF, Yen YS (2011) TFEID: Taiwanese facial expression image database, November

  22. Yu J, Bhanu B (2006) Evolutionary feature synthesis for facial expression recognition. Pattern Recognit Lett 27(11):1289–1298

    Article  Google Scholar 

  23. Littlewort G et al (2006) Dynamics of facial expression extracted automatically from video. Image Vis Comput 24(6):615–625

    Article  Google Scholar 

  24. Feng X, Pietikainen M, Hadid A (2007) Facial expression recognition with local binary patterns. Pattern Recognit Image Anal 17(4):592–598

    Article  Google Scholar 

  25. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: a comprehensive study. Image Vis Comput 27(6):803–816

    Article  Google Scholar 

  26. Xie X, Lam KM (2009) Facial expression recognition based on shape and texture. Pattern Recognit 42(5):1003–1011

    Article  Google Scholar 

  27. Lajevardi SM, Hussain ZM (2009) Zernike moments for facial expression recognition. In: Proceedings of International conference on communication, computer and power, pp 15–18

  28. Yang P, Liu Q, Metaxas D (2010) Exploring facial expressions with compositional features. In: Proceedings of IEEE conference on computer vision and pattern recognition, pp 2638–2644

  29. Zhi R, Flierl M, Ruan Q, Kleijn W (2011) Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition. IEEE Trans Syst Man Cybern Part B Cybern 41(1):38–52

    Article  Google Scholar 

  30. Cotter SF (2010) Sparse representation for accurate classification of corrupted and occluded facial expressions. In: Proceedings of the international conference on acoustics speech and signal processing, pp 838–841

  31. Kobayashi H, Hara F (1992) Recognition of mixed facial expressions and their strength by neural network. In: Proceedings of IEEE international workshop on robot and human communication, pp 381–386

  32. Ushida H, Takagi T, Yamaguchi T (1993) Recognition of facial expressions using the conceptual fuzzy sets. In: Proceedings of the 2nd international conference on fuzzy systems, pp 594–599

  33. Sohail ASM, Bhattacharya P (2007) Classification of facial expressions using k-nearest neighbor classifier. In: Advances in computer vision and computer graphics. Lecture notes in computer science. 4418:555–566

  34. Cheng S, Chen M, Chang H, Chao T (2007) Semantic-based facial expression recognition using analytical hierarchy process. Expert Syst Appl 33(1):86–95

    Article  Google Scholar 

  35. Yang P, Liu Q, Metaxas D (2011) Dynamic soft encoded patterns for facial event analysis. Comput Vis Image Underst 115(3):456–465

    Article  Google Scholar 

  36. Buenaposada JM, Muñz E, Baumela L (2008) Recognising facial expressions in video sequences. Pattern Anal Appl 11(2-3):101–116

    Article  Google Scholar 

  37. Bolme D, Teixeira M, Beveridge J, Draper B (2003) The CSU face identification evaluation system user’s guide: its purpose, feature and structure. In: Proceedings of the 3rd international conference on computer vision systems, pp 304–313

  38. Wong KW, Lam KM, Siu WC (2001) An efficient algorithm for human face detection and facial feature extraction under different conditions. Pattern Recognit 34(10):1993–2004

    Article  MATH  Google Scholar 

  39. Wolpert DH (1992) Stacked generalization. Neural Netw 5:241–259

    Article  Google Scholar 

  40. Wu T-F, Lin C-J, Weng RC (2003) Probability estimates for multiclass classification by pairwise coupling. J Mach Learn Res 5:975–1005

    MathSciNet  Google Scholar 

  41. Ahonen T, Hadid A, Pietik M (2004) Face recognition with local binary patterns. In: Proceedings of the 8th European conference on computer vision, pp 469–481

  42. Daugman JG (1985) Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. J Opt Soc Am A 2(7):1160–1169

    Article  Google Scholar 

  43. Lyons MJ, Akamatsu S, Kamachi M, Gyoba J (1998) Coding facial expressions with Gabor wavelets. In: Proceedings of the 3rd IEEE international conference on automatic face and gesture recognition, pp 200–205

  44. Shen L, Bai L (2006) A review on gabor wavelets for face recognition. Pattern Anal Appl 9(2-3):273–292

    Article  MathSciNet  Google Scholar 

  45. Bartlett MS, Littlewort G, Frank M, Lainscsek C, Fasel I, Movellan J (2005) Recognizing facial expression: machine learning and application to spontaneous behavior. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 568–573

  46. Donato G, Bartlett MS, Hager J, Ekman P, Sejnowski T (1999) Classifying facial actions. IEEE Trans Pattern Anal Mach Intell 21(10):974–989

    Article  Google Scholar 

  47. Farajzadeh N, Faez K, Pan G (2010) Study on the performance of moments as invariant descriptors for practical face recognition systems. IET Comput Vis 4(4):272–285

    Article  Google Scholar 

  48. Guo G, Dyer CR (2005) Learning from examples in the small sample case: face expression recognition. IEEE Trans Syst Man Cybern Part B Cybern 35(3):477–488

    Article  Google Scholar 

  49. Crammer K, Singer Y (2001) On the algorithmic implementation of multiclass svms. J Mach Learn Res 2:265–292

    Google Scholar 

  50. Chang CC, Lin CJ (2011) LIBSVM: a library for support vector machines

  51. Hsu CW, Chang CC, Lin CJ (2003) A practical guide to support vector classification. Technical Report, Department of Computer Science, National Taiwan University

  52. Snyman JA (2005) Practical mathematical optimization: an introduction to basic optimization theory and classical and new gradient-based algorithms. Springer, Berlin

  53. Donoho DL (2006) Compressed sensing. IEEE Trans Inf Theory 52(4):1289–1306

    Article  MathSciNet  MATH  Google Scholar 

  54. Candes EJ, Romberg J, Tao T (2006) Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans Inf Theory 52(2):489–509

    Article  MathSciNet  MATH  Google Scholar 

  55. Tsaig Y, Donoho D, Stodden V (2007) Sparselab, May

  56. Kohonen T (1997) Self organizing maps, 2nd edn. Springer, Berlin

  57. Iman RL, Davenport JM (1980) Approximations of the critical region of the friedman statistic. Commun Stat A9:571–595

    Article  MATH  Google Scholar 

  58. Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6(2):65–70

    MathSciNet  MATH  Google Scholar 

  59. Sommer D, Golz M (2002) Multiple training of vector-based neural networks to detect density centers in input space. In: Proceedings of the European symposium on intelligent technologies, hybrid systems and their implementation on smart adaptive systems, pp 135–137

  60. Geman S, Bienenstock E, Doursat R (1992) Neural networks and the bias/variance dilemma. Neural Comput 4:1–58

    Article  Google Scholar 

  61. Wang C, Guo C (2010) Face recognition based on gabor-enhanced manifold learning and SVM. In: Advances in neural networks. Lecture notes in computer science. 6064:184–191

  62. Zhi R, Ruan Q (2009) Robust facial expression recognition using selected wavelet moments invariants. In: Proceedings of the global congress on intelligent system, pp 508–512

Download references

Acknowledgements

We would like to thank Prof. Guodong Guo for providing us the facial fiducial points of JAFFE dataset. This work is partly supported by NSF of China (No. 61070067) and National 973 Program (2013CB329504).

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Zhejiang University, Hangzhou, 310027, China

    Nacer Farajzadeh, Gang Pan & Zhaohui Wu

Authors
  1. Nacer Farajzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gang Pan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhaohui Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Nacer Farajzadeh or Gang Pan.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Farajzadeh, N., Pan, G. & Wu, Z. Facial expression recognition based on meta probability codes. Pattern Anal Applic 17, 763–781 (2014). https://doi.org/10.1007/s10044-012-0315-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-012-0315-5

Keywords

Search

Navigation