Skip to main content
SpringerLink
Log in

Facial expression recognition based on image pyramid and single-branch decision tree

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this paper, a new approach has been proposed for improved facial expression recognition. The new approach is inspired by the compressive sensing theory and multiresolution approach to facial expression problems. Initially, each image sample is decomposed into desired pyramid levels at different sizes and resolutions. Pyramid features at all levels are concatenated to form a pyramid feature vector. The vectors are further reinforced and reduced in dimension using a measurement matrix based on compressive sensing theory. For classification, a multilevel classification approach based on single-branch decision tree has been proposed. The proposed multilevel classification approach trains a number of binary support vector machines equal to the number of classes in the datasets. Class of test data is evaluated through the nodes of the tree from the root to its apex. The results obtained from the approach are impressive and outperform most of its counterparts in the literature under the same databases and settings.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Fasel, B., Juergen, L.: Automatic facial expression analysis. J. Pattern Recognit. Soc. 36, 259–275 (2003)

    Article  MATH  Google Scholar 

  2. Min, T., Feng, C.: Facial expression recognition and its application based on Curvelet transform and PSO_SVM. Int. J. Light Electron Opt. 124, 5401–5406 (2013)

    Article  Google Scholar 

  3. Wenfei, G., Cheng, X., Venkatesh, Y.V., Dong, H., Hai, L.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. J. Pattern Recognit. Soc. 45, 80–91 (2012)

    Article  Google Scholar 

  4. Shiqing, Z., Lemin, L., Zhijin, Z.: Facial expression recognition based on Gabor wavelets and sparse representation. In: Proceedings ICSP, pp. 816–819 (2012)

  5. Michael, J.L., Shigeru, A., Miyuki K., Jiro, G.: Coding facial expressions with Gabor wavelets. In: Proceedings AFGR, pp. 200–205 (1998)

  6. Shishir, B., Ganesh, K.V.: Recognition of facial expressions using Gabor wavelets and learning vector quantization. J. Eng. Appl. Artif. Intell. 21, 1056–1064 (2008)

    Article  Google Scholar 

  7. Baochang Z., Shiguang, S.: Histogram of Gabor phase patterns (HGPP): a novel object representation approach for face recognition. In: IEEE Transaction IP, pp. 57–68 (2007)

  8. Yimo, G., Zhengguang, X.: Local Gabor phase difference pattern for face recognition. In: Proceedings ICPR, pp. 1–4 (2008)

  9. Rafael, C.G., Richard, E.W.: Wavelets and Multiresolution Processing, In Digital Image Processing 2nd Ed., Pearson publishers, USA

  10. Shannon, C.E.: Communication in the presence of noise. Proc. IRE 37, 10–21 (1949)

    Article  MathSciNet  Google Scholar 

  11. Emamnuel, C.J.: Compressive Sampling. Proc. ICM 3, 1433–1452 (2006)

    MathSciNet  MATH  Google Scholar 

  12. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 27, 379–423 (1948)

    Article  MathSciNet  MATH  Google Scholar 

  13. Baraniuk, R.G.: Compressed sensing [lecture notes]. Proc. SPM 24, 118–124 (2007)

    Google Scholar 

  14. Eleyan, A., Kose, K., Cetin, E.: Image feature extraction using compressive sensing. In: proceedings AISC, pp. 177–184 (2013)

  15. Hsu, C., Lin, C.: A comparison of methods for multi-class support vector machines. IEEE Trans. Neural Network 13, 415–425 (2002)

    Article  Google Scholar 

  16. Vapnik, V.: Statistical Learning Theory. Wiley, New York (1998)

    MATH  Google Scholar 

  17. Knerr, S., Personnaz, L., Dreyfus, G.: “Single-layer learning revisited: a stepwise Procedure for Building and Training a Neural Network”. In: Fogelman-Soulie and Herault, (eds), Neurocomputing: Algorithms, Architectures and Applications NATO ASI. Springer, (1990)

  18. Friedman. J. H.: Another approach to polychotomous classification. Technical report. Stanford Department of Statistics. http://www.stat.stanford.edu/reports/friedman/poly.ps.Z (1996)

  19. Platt, J. C., Cristianini N., Shawe-Taylor, J.: Large margin DAGs for multiclass classification. http://www.weizmann.ac.il (1999)

  20. Dietterich, T.G., Bakiri, G.: Solving multiclass learning problems via error-correcting output codes. J. Artif. Intell. Res. 2, 263–286 (1995)

    MATH  Google Scholar 

  21. Mlakar, U., Potocnik, B.: Automated facial expression recognition based on histograms of oriented gradient feature vector differences. In: Proceedings of SIVP, Vol. 9(1), pp. 245–253

  22. Yurtkan, K., Demirel, H.: Entropy-based feature selection for improved 3D facial expression recognition. Proceedings of SIVP 8(2), 267–277 (2014)

    Google Scholar 

  23. Mohammadian, A., Aghaeinia, H., Towhidkhah, F.: Incorporating prior knowledge from the new Person into recognition of facial expression. Proceedings of SIVP 10(2), 235–242 (2016)

    Google Scholar 

  24. Zhalehpour, S., Akhtar, Z., Eroglu, E.: Multimodal emotion recognition based on Peak frame selection from video. Proceedings of SIVP 10(5), 827–834 (2016)

    Google Scholar 

  25. Rao, K.S., Koolagudi, S.G.: Recognition of emotions from video using acoustic and facial features. Proceedings of SIVP 9(5), 1029–1045 (2015)

    Google Scholar 

  26. Moeini, A., Faez K., Moeini, H.: Pose-invariant-facial expression recognition based on 3D face reconstruction and synthesis from a single 2D image. In: Proceedings 22nd International Conference on ICPR (2014), pp. 1746–1751 (2014)

  27. Moeini, A., Faez, K., Moeini, H.: Multimodal Facial Expression Recognition Based on 3D Face Reconstruction from 2D Images. In: Ji, Qiang, Gang Hua, Thomas B.Moeslund, Nasrollahi, Kamal (eds.) Face and facial expression recognition from real world videos. International Workshop, Stockholm (2014)

    Google Scholar 

  28. Ouamane, A., Belahcene, M., Benakcha, A., et al.: Robust multimodal 2D and 3D face authentication using local feature fusion. Proc. SIVP 10(1), 129–137 (2016)

    Google Scholar 

  29. Lajevardi, S.M.: Structural similarity classifier for facial expression recognition. Proc. SIVP 8(6), 1103–1110 (2014)

    Google Scholar 

  30. Okuwobi, I.P., Chen, Q., Niu, S., Bekalo, L.: Three-dimensional (3D) facial recognition and prediction. Proc. SIVP 10(6), 1151–1158 (2016)

    Google Scholar 

  31. Liang, J., Hou, Z., Chen, C., Xu, X.: Supervised bilateral two-dimensional locality preserving projection algorithm based on Gabor wavelet. In: Proceedings SIVP, pp. 1–8 (2016)

  32. Khan, N.M., Ksantini, R., Ahmad, I.S., Guan, L.: SN-SVM: a sparse nonparametric support vector machine classifier. Proc. SIVP 8(8), 1625–1637 (2014)

    Google Scholar 

  33. Wang, Z., Xu, W., Hu, J., Guo, J.: A Multiclass SVM method via probabilistic error-correcting output codes. In: Proceedings ITA, pp. 1–4 (2010)

  34. Chen, L., Yen, Y.: Taiwanese Facial Expression Image Database. Brain Mapping Laboratory, Institute of Brain Science, National Yang-Ming University, Taipei (2007)

    Google Scholar 

  35. Kanade, T., Cohn, J.F., Tian, Y.: Comprehensive database for facial expression analysis. In: Proceedings FG2000-ICAFGR, pp. 46–53. Grenoble, France (2000)

  36. Wei-Lu, C., Jian-Jiun, D., Jun-Zuo, L.: Facial expression recognition based on improved local binary pattern and class-regularized locality preserving projection. Int. J. Signal Process. 117, 1–10 (2015)

    Article  Google Scholar 

  37. Ying, Z., Fang, X.: Combining LBP and adaboost for facial expression recognition\(^{\prime \prime }\). In: Proceedings ICSP, pp. 1461–1464 (2008)

  38. Guo, G., Dyer, R.: Facial expression recognition based on Gabor histogram feature and MVBoost. J. Comput. Res. Dev 44, 1089–1096 (2007)

    Article  Google Scholar 

  39. Huang, M. W., Wang, Z. W., Ying, Z.L.: A new method for facial expression recognition based on sparse representation plus LBP. In: Proceedings ICISP, pp. 1750–1754 (2010)

  40. Cai L., Yin, Z.: A new approach of facial expression recognition based on Contourlet transform. In: Proceedings ICWAPR, pp. 275–280 (2009)

  41. Zavaschi, T., Koerich, A., Oliveira, L.: Facial expression recognition using ensemble of classifiers. In: Proceedings IC-ASSP, pp. 1489–1492 (2011)

  42. Shan, C., Gong, S., Facial expression analysis across databases. In: Proceedings MT, pp. 317–320 (2011)

  43. Zhang, Z., Xu, C., Wang, J.X., Chen, X.N.: Facial expression recognition based on MB-LGBP feature and multi-level classification. J. Adv. Intell. Soft. Comput. 129, 37–42 (2012)

    Google Scholar 

  44. Moeini, A., Faez, K., Sadeghi, H., Moeini, H.: 2D facial expression recognition via 3D reconstruction and feature fusion. J. Vis. Commun. Image R 35, 1–14 (2016)

    Article  Google Scholar 

  45. Rivera, A.R., Castillo, J.R., Chae, O.: Local directional number pattern for face analysis: face and expression recognition. IEEE Trans. Image Process. 22(5), 1740–1752 (2013)

    Article  MathSciNet  Google Scholar 

  46. Liu, S., Ruan, Q., Wang, C., An, G.: Tensor rank one differential graph preserving analysis for facial expression recognition. Image Vis. Comput. 30, 535–545 (2012)

    Article  Google Scholar 

  47. Shan, C., Gong, S., McOwan, P.W.: Facial expressionrecognition based on local binary patterns: a comprehensive study. Image Vis. Comput. 27(6), 803–816 (2009)

    Article  Google Scholar 

  48. Gu, W., Xiang, C., Venkatesh, Y.V., Huang, D., Lin, H.: Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit. 45, 80–91 (2012)

    Article  Google Scholar 

  49. Yeasin, M., Bullot, B., Sharma, R.: From facial expression to level of interest: a Spatio-temporal approach. In: Proceedings CVPR, pp. 922–927 (2004)

  50. Aleksic, P. S., Katsaggelos, A. K.: Automatic facial expression recognition using facial animation parameters and ,multi-stream HMMS. In: Transactions IFS, pp. 3–11 (2006)

  51. Li, Z. S., Imai, J., Kaneko, M.: Facial expression recognition using facial-component-based bag of words and PHOG descriptors. In: proceedings IMT, ,pp. 1003–1009 (2010)

  52. Shan, C., Gong, S., McOwan, P.W.: Robust facial expression recognition using local binary patterns. In: Proceedings ICIP, pp. 370–373 (2005)

  53. Zhao, G.Y., Pietik, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. In Transactions PAMI, pp. 915–928 (2007)

  54. Bartlett, M. S., Littlewort, G., Fasel, I., Movellan, R.: Real-time face detection and facial expression recognition: development and application to human computer interaction. In: workshop on HCI-CVPR (2003)

  55. Littlewort, G., Bartlett, M., Fasel, I., Susskind J., Movellan, J.: Dynamics of facial expression extracted automatically from video. In: Workshop on Face Processing in Video (2004)

  56. Tian, Y.: Evaluation of face resolution for expression analysis. In: Workshop on Face Processing in Video (2004)

  57. Rudovic, O., Pavlovic, V., Pantic, M.: Multi-output laplacian dynamic ordinal regression for facial expression recognition and intensity estimation. In: Proceedings International Conference on CVPR, 26 July, pp. 2634–2641 (2012)

  58. Zhong, L., Liu, Q., Yang, P., Liu, B., Huang, J., Metaxas, D. N.: Learning active facial patches for expression analysis. In: Proceedings International Conference On CVPR, 26 July, pp. 2634–2641 (2012)

Download references

Author information

Authors and Affiliations

  1. Department of Electric and Electronic Engineering, Selçuk University, Konya, Turkey

    Abubakar M. Ashir

  2. Department of Electric and Electronic Engineering, Avrasya University, Trabzon, Turkey

    Alaa Eleyan

Authors
  1. Abubakar M. Ashir
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alaa Eleyan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abubakar M. Ashir.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ashir, A.M., Eleyan, A. Facial expression recognition based on image pyramid and single-branch decision tree. SIViP 11, 1017–1024 (2017). https://doi.org/10.1007/s11760-016-1052-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-016-1052-9

Keywords

Search

Navigation