Skip to main content

Advertisement

SpringerLink
Log in

Facial expression recognition based on a mlp neural network using constructive training algorithm

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

This paper presents a constructive training algorithm for Multi Layer Perceptron (MLP) applied to facial expression recognition applications. The developed algorithm is composed by a single hidden-layer using a given number of neurons and a small number of training patterns. When the Mean Square Error MSE on the Training Data TD is not reduced to a predefined value, the number of hidden neurons grows during the neural network learning. Input patterns are trained incrementally until all patterns of TD are presented and learned. The proposed MLP constructive training algorithm seeks to find synthesis parameters as the number of patterns corresponding for subsets of each class to be presented initially in the training step, the initial number of hidden neurons, the number of iterations during the training step as well as the MSE predefined value. The suggested algorithm is developed in order to classify a facial expression. For the feature extraction stage, a biological vision-based facial description, namely Perceived Facial Images PFI has been applied to extract features from human face images. To evaluate, the proposed approach is tested on three databases which are the GEMEP FERA 2011, the Cohn-Kanade facial expression and the facial expression recognition FER-2013 databases. Compared to the fixed MLP architecture and the literature review, experimental results clearly demonstrate the efficiency of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Banziger T, Scherer KR (2010) Introducing the geneva multimodal emotion portrayal (gemep) corpus, Blueprint for Affective Computing, A Sourcebook, Series in affective science, chapter 6.1 pp 271–294. Oxford University Press, Oxford

    Google Scholar 

  2. Bejani M, Gharavian D, Charkari NM (2012) method, Audiovisual emotion recognition using ANOVA feature selection networks, multi-classifier neural Neural Computer & Application. Springer-Verlag London

  3. Boughrara H, Chen L, Ben Amar C, Chtourou M (2012) Face Recognition under varying Facial Expression Based on Perceived Facial Images And Local Feature Matching, International Conference on Information Technology and e-Service, pp 1–6, 2nd edition. Sousse Tunisia

  4. Boughrara H, Chtourou M, Ben Amar C (2012) MLP Neural Network Based Face Recognition System Using Constructive Training algorithm, International Conference on Multimedia Computing and System ICMCS, pp 233–238, 3rd edition. Tangier Morocco

  5. Boughrara H, Chen L, Ben Amar C, Chtourou M (2013) Facial Expression Recognition Based on Perceived Facial Images and Local Feature Matching, International Conference on Image Analysis and Processing, ICIAP, Part II, LNCS 8157. Springer-Verlag Berlin, Heidelberg, pp 591–600

    Google Scholar 

  6. Chakrabarti D, Dutta D (2013) Facial Expression Recognition Using Eigenspaces Procedia Technology vol 10, pp 755–761

  7. Chumkamon S, Hayashi E (2013) Facial expression recognition using constrained local models and Hidden Markov models with consciousness-based architecture IEEE/SICE International Symposium on System Integration (SII), pp 382–387

  8. Dahmane M, Meunier J (2011) Emotion recognition using dynamic gridbased hog features IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 884–888

  9. Danisman T, Bilasco IM, Martinet J, Djeraba C (2013) Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron. Signal Process 93(6):1547–1556

    Article  Google Scholar 

  10. Danisman T, Bilasco IM, Martinet J, Djeraba C (2013) Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron. Signal Process 93(6):1547–1556

    Article  Google Scholar 

  11. Danisman T, Bilasco IM, Martinet J, Djeraba C (2013) Intelligent pixels of interest selection with application to facial expression recognition using multilayer perceptron. Signal Process 93(6):1547– 1556

    Article  Google Scholar 

  12. Dhall A, Asthana A, Goecke R, Gedeon T (2011) phog Emotion recognition using, features, lpq IEEE Int’l Conf Automatic Face and Gesture Analysis, pp 878–883

  13. Edelman S, Intrator N, Poggio T (1997) Complex cells and object recognition, unpublished manuscript. http://kybele.psych.cornell.edu/edelman/archive.html

  14. Ekman P (1992) Facial expressions of emotion: an old controversy and new findings. Philos Trans: Biol Sci 335:63–69

    Article  Google Scholar 

  15. Eskil MT, Benli KS (2014) Facial expression recognition based on anatomy Computer Vision and Image Understanding vol 119, pp 1–14

  16. Fan W, Bouguila N (2013) Face detection and facial expression recognition using simultaneous clustering and feature selection via an expectation propagation statistical learning framework Multimed Tools Appl

  17. Fang H, Parthalïn NM, Aubrey AJ, Tam G KL, Borgoa R, Rosin PL, Grant PW, Marshall D, Chend M (2014) Facial expression recognition in dynamic sequences: An integrated approach. Pattern Recognit 47(3):1271–1281

    Article  Google Scholar 

  18. Farajzadeh N, Pan G, Wu Z (2013) Facial expression recognition based on meta probability codes, Pattern Analysis Application. Springer-Verlag London

  19. Gizatdinova Y, Surakka V (2006) Feature-based detection of facial landmarks from neutral and expressive facial images. IEEE Trans Pattern Anal Mach Intell 28:135139

    Article  Google Scholar 

  20. Gu W, Xiang C, Venkatesh YV, Huang D, Lin H (2012) Facial expression recognition using radial encoding of local Gabor features and classifier synthesis. Pattern Recognit 45:80–91

    Article  Google Scholar 

  21. Honggui H, Junfei Q. (2009) A Novel Pruning Algorithm for Self-organizing Neural Network, Proceedings of International Joint Conference on Neural Networks, Atlanta, Georgia USA, pp 1245–1250

  22. Huang D, Ben Soltana W, Ardabilian M, Wang YH, Chen L (2011) Textured 3D face recognition using biological vision-based facial representation and optimized weighted sum fusion Proc. CVPR Workshop on Biometrics, Colorado Springs, CO USA

  23. Huang D (2011) Robust Face Recognition Based on Three Dimentional Data,thesis informatic, LRIS. university of Lyon

  24. Jaimes A, Sebe N (2007) Multimodal human-computer interaction: a survey. Comput Vis Image Underst 108:116–134

    Article  Google Scholar 

  25. Kaburlasos VG, Papadakis SE, Papakostas GA (2013) Lattice computing extension of the FAM neural classifier for human facial expression recognition. IEEE Trans Neural Netw Learn Syst 24(10):1526–1538

    Article  Google Scholar 

  26. Kanade T, Cohn J, Tian YL (2000) Comprehensive database for facial expression analysis, International Conference on Automatic Face and Gesture Recognition, 4th edition, pp 46–53

  27. Khan RA, Meyer A, Konik H, Bouakaz S (2013) Framework for reliable real-time facial expression recognition for low resolution images. Pattern Recognit Lett 34:11591168

    Article  Google Scholar 

  28. Lajevardi SM, Hussain ZM (2012) Automatic facial expression recognition: feature extraction and selection. SIViP 6:159–169

    Article  Google Scholar 

  29. Lee SH, Kim H, Man Ro Y, Plataniotis KN (2013) Using color texture sparsity for facial expression recognition, 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pp 1–6

  30. HC, Wu CY, Lin TM (2013) Facial Expression Recognition Using Image Processing Techniques and Neural Networks Advances in Intelligent Systems & Applications. Springer-Verlag Berlin Heidelberg, pp 259–267

  31. Lee C, Huang S, Shih C (2010) Facial affect recognition using regularized discriminant analysis-based algorithms, EURASIP J Adv Signal Process vol 10

  32. Lienhart R, Maydt J (2001) An extended set of haar-like features for rapid object detection. Int Conf Image Process 1:900–903

    Article  Google Scholar 

  33. Liu S, Ruan Q, Wang C, An G (2012) Tensor rank one differential graph preserving analysis for facial expression recognition. Image Vis Comput 30:535–545

    Article  Google Scholar 

  34. Liu S, Ruan Q, Wang C, An G (2012) Tensor rank one differential graph preserving analysis for facial expression recognition. Image Vis Comput 30:535–545

    Article  Google Scholar 

  35. Liu D, Tsu-Shuan C, Zhang Y (2002) A constructive algorithm for feedforward neural networks with incremental training. Trans Circ Syst-Fundam Theory Appl 49:1876–1879

    Article  Google Scholar 

  36. Lippmann RP (1989) Pattern classification using neural networks, IEEE Communications Magazine, vol 27, pp 47–50 59–64

  37. Littlewort G, Whitehill J, Wu T, Butko N, Ruvolo P, Movellan J, Bartlett M (2011) The motion in emotion-a cert based approach to the fera emotion challenge IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 897–902

  38. Liu S, Ruan Q, Wang C, An G (2012) Tensor rank one differential graph preserving analysis for facial expression recognition. Image Vis Comput 30:535–545

    Article  Google Scholar 

  39. Long F, Wu T, Movellan JR, Bartlett MS, Littlewort G (2012) Learning spatiotemporal features by using independent component analysis with application to facial expression recognition. Neurocomputing 93:126132

    Article  Google Scholar 

  40. Ma L, Khorasani K (2004) Facial expression recognition using constructive feedforward neural networks. IEEE Trans Syst Man Cybern B: Cybern 34:1588–1595

    Article  Google Scholar 

  41. Masmoudi S, Frikha M, Chtourou M, Hamida A, Efficient MLP (2011) constructive training algorithm using a neuron recruiting approach for isolated word recognition system. Int J Speech Technol 14:1–10

    Article  Google Scholar 

  42. Meng H, Romera-Paredes B, Berthouze N (2011) Emotion recognition by two view svm 2k classifier on dynamic facial expression features IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 854–859

  43. Majumder A, Behera L, Subramanian VK (2014) Emotion recognition from geometric facial features using self-organizing map. Pattern Recognit 47 (3):1282–1293

    Article  Google Scholar 

  44. Mohseni S, Kordy HM, Ahmadi R (2013) Facial expression recognition using DCT features and neural network based decision tree, 55th International Symposium ELMAR, pp 361–364

  45. Niese R, Al-Hamadi A, Farag A, Neumann H, Michaelis B (2012) Facial expression recognition based on geometric and optical flow features in colour image sequences. IET Comput Vis 6(2):79–89

    Article  MathSciNet  Google Scholar 

  46. Owusu E, Zhan Y, Mao QR (2014) A neural-AdaBoost based facial expression recognition system. Expert Syst Appl 41(7):3383–3390

    Article  Google Scholar 

  47. Pratama M, Anavatti SG, Angelov PP, Lughofer E (2014) PANFIS: A novel incremental learning machine. IEEE Trans Neural Netw Learn Syst 25(1):55–68

    Article  Google Scholar 

  48. Puma-Villanueva WJ, dos Santos EP, Von Zuben FJ (2012) A constructive algorithm to synthesize arbitrarily connected feedforward neural networks. Neurocomputing 75(1):14–32

    Article  Google Scholar 

  49. Sadeghi H, Raie A-A, Mohammadi M-R (2013) Facial expression recognition using geometric normalization and appearance representation, 8th Iranian Conference on Machine Vision and Image Processing (MVIP), pp 159–163

  50. Sharma SK, Chandra P (2010) Constructive neural networks: A Review. Int J Eng Sci Technol 2(12):7847–7855

    Google Scholar 

  51. Shan C, Gong S, McOwan PW (2009) Facial expression recognition based on local binary patterns: A comprehensive study. Image Vis Comput 27:803–816

    Article  Google Scholar 

  52. Srivastava R, Roy S, Yan S, Sim T (2011) Accumulated motion images for facial expression recognition in videos IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 903–908

  53. Sridhar SS, Ponnavaikko M (2011) Improved adaptive learning algorithm for constructive neural networks. Int J Comput Electr Eng 3 (1):1793–8163

    Google Scholar 

  54. https://www.kaggle.com/c/challenges-in-representation-learning-facial-expression-recognition-challenge/data

  55. Tang Y (2013) Machines, Deep learning using linear support vector Workshop on Challenges in Representation Learning ICML

  56. Tariq U, Lin K-H, Li Z, Zhou X, Wang Z, Le V, Huang T, Lv X, Han T (2011) Emotion recognition from an ensemble of features IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 872–877

  57. Tian Y, Kanade T, Cohn J (2005) Facial expression analysis Handbook of Face Recognition. Springer. (Chapter 11)

  58. Ionescu RT, Popescu M, Grozea C (2013) Local learning to improve bag of visual words model for facial expression recognition. Workshop on Challenges in Representation Learning ICML

  59. Valstar MF, Jiang B, Mehu M, Pantic M, Scherer K (2011) The First Facial Expression Recognition and Analysis Challenge, 9th IEEE conference on Automatic Face and Gesture Recognition. Santa Barbara, California

    Google Scholar 

  60. Villegas M, Paredes R (2011) Dimensionality reduction by minimizing nearest-neighbor classification error. Pattern Recognit Lett 32:633–639

    Article  Google Scholar 

  61. Viola P , Jones M (2004) Robust real-time face detection. Int J Comput Vis 57(2):137154

    Article  Google Scholar 

  62. Wan S, Aggarwal JK (2014) Spontaneous facial expression recognition: A robust metric learning approach. Pattern Recognit 47(5):1859–1868

    Article  Google Scholar 

  63. Wang Z, Ruan Q, An G (2012) Facial expression recognition based on tensor local linear discriminant analysis IEEE 11th International Conference on Signal Processing (ICSP), pp 1226–1229

  64. Wang S, Liu Z, Wang Z, Wu G, Shen P, He S, Wang X (2013) Analyses of a multimodal spontaneous facial expression database. IEEE Trans Affect Comput 4(1):34–46

    Article  Google Scholar 

  65. Luo Y, Wu C (2013) Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik - Int J Light Electron Opt 124(17):2767–2770

    Article  Google Scholar 

  66. Yang P, Liu Q, Metaxas DN (2010) Exploring facial expressions with compositional features. IEEE Conf Comput Vis Pattern Recognit:2638–2644

  67. Yang S, Bhanu B (2011) Facial expression recognition using emotion avatar image IEEE Int’l Conf. Automatic Face and Gesture Analysis, pp 866–871

  68. Zhang Z, Qiao J (2010) A Node Pruning Algorithm for Feedforward Neural Network Based on Neural Complexity, International Conference on Intelligent Control and Information Processing Dalian, China, pp 406–410

  69. Zhao X, Zhang S (2012) Facial expression recognition using local binary patterns and discriminant kernel locally linear embedding. EURASIP J Adv Signal Process 10:1687–6180

    Google Scholar 

  70. Zavaschi THH, Britto Jr AS, Oliveira LES, Koerich AL (2013) Fusion of feature sets and classifiers for facial expression recognition. Expert Syst Appl 40(2):646–655

    Article  Google Scholar 

  71. Zhen W, Zilu Y (2012) Facial Expression Recognition Based on Local Phase Quantization and Sparse Representation Eighth International Conference on Natural Computation (ICNC), pp 222–225

Download references

Author information

Authors and Affiliations

  1. Control and Energy Management Laboratory, University of Sfax, ENIS, BP 1173, Sfax, 3038, Tunisia

    Hayet Boughrara & Mohamed Chtourou

  2. Research Groups on Intelligent Machines, University of Sfax, ENIS, BP 1173, Sfax, 3038, Tunisia

    Chokri Ben Amar

  3. Laboratoire d’InfoRmatique en Image et Systémes d’information, University of Lyon, 69134, Ecully, France

    Liming Chen

Authors
  1. Hayet Boughrara
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohamed Chtourou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chokri Ben Amar
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hayet Boughrara.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Boughrara, H., Chtourou, M., Ben Amar, C. et al. Facial expression recognition based on a mlp neural network using constructive training algorithm. Multimed Tools Appl 75, 709–731 (2016). https://doi.org/10.1007/s11042-014-2322-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-014-2322-6

Keywords

Search

Navigation