Abstract
Facial expression is the most common technique is used to convey the expressions of human beings. Due to different ethnicity and age, faces differ from one individual to another so that an automatic facial expression analysis and recognition is a difficult operation. To solve this difficulty, this paper proposes a robust feature extraction with optimized DBN-SMO for facial expression recognition (FER). Initially, the pre-processing stage is performed then texture descriptors of Local Phase Quantization (LPQ), Weber Local Descriptor (WLD) and Local Binary Pattern (LBP) are used to extract the features. Moreover, Discrete Cosine Transform (DCT) features are extracted to enhance the recognition rate and reduce the computational cost. After that, the Principal component analysis (PCA) is used for dimension reduction. Finally, a deep belief network (DBN) with Spider monkey optimization (SMO) algorithm is used to classify basic expressions for FER. Here, SMO algorithm is used to optimize bias factors and initial connection weights that control the efficiency of the DBN. The proposed work is performed in the MATLAB environment. Experiments performed on Karolinska Directed Emotional Faces (KDEF), Man-Machine Interaction (MMI), Cohn Kanade (CK+), Extended Denver Intensity of Spontaneous Facial Actions (DISFA+) and Oulu-Chinese Academy of Science Institute of Automation (Oulu-CASIA) datasets and it provides a classification accuracy of 97.93%, 95.42%, 97.58%, 95.76%, and 92.38% respectively, this is higher than other current procedures for seven-class emotion.
Similar content being viewed by others
References
Ali K and Hughes CE (2019) Facial expression recognition using disentangled adversarial learning. arXiv preprint arXiv:1909.13135
Alphonse SA, Dharma D (2018) Novel directional patterns and a generalized supervised dimension reduction system (GSDRS) for facial emotion recognition. Multimed Tools Appl 77(8):9455–9488
Alrjebi MM, Liu W, Li L (2018) Face recognition against illuminations using two directional multi-level threshold-LBP and DCT. Multimed Tools Appl:1–21
Chen J, Lv Y, Xu R, Xu C (2019) Automatic social signal analysis: facial expression recognition using difference convolution neural network. Journal of Parallel and Distributed Computing 131:97–102
Dabhi MK, Pancholi BK (2016) Face detection system based on viola-jones algorithm. International Journal of Science and Research (IJSR) 5(4):62–64
Eng KS, Ali H, Cheah AY, Chong YF (2019) Facial expression recognition in JAFFE and KDEF datasets using histogram of oriented gradients and support vector machine. In IOP conference series: materials science and engineering. IOP Publishing 705(1):012031
Fan X, Tjahjadi T (2017) A dynamic framework based on local Zernike moment and motion history image for facial expression recognition. Pattern Recogn 64:399–406
Fang Y, Chang L (2015) Multi-instance feature learning based on sparse representation for facial expression recognition. In: International conference on multimedia modeling. Springer, Cham, pp 224–233
Georgescu M-I, Ionescu RT, Popescu M (2019) Local learning with deep and handcrafted features for facial expression recognition. IEEE Access 7:64827–64836
Ijjina EP and Mohan CK (2014) Facial expression recognition using Kinect depth sensor and convolutional neural networks. In machine learning and applications (ICMLA), 2014 13th international conference on IEEE, 392-396
Jain DK, Zhang Z and Huang K (2017) Multi angle optimal pattern-based deep learning for automatic facial expression recognition. Pattern Recogn Lett
Ji Y, Hu Y, Yang Y, Shen F, Shen HT (2019) Cross-domain facial expression recognition via an intra-category common feature and inter-category distinction feature fusion network. Neurocomputing 333:231–239
Jiang R, Ho ATS, Cheheb I, Al-Maadeed N, Al-Maadeed S, Bouridane A (2017) Emotion recognition from scrambled facial images via many graph embedding. Pattern Recogn 67:245–251
Jung H, Lee S, Yim J, Park S and Kim J (2015) Joint fine-tuning in deep neural networks for facial expression recognition, in the IEEE international conference on computer vision (ICCV), IEEE
Khan SA, Hussain A, Usman M (2018) Reliable facial expression recognition for multi-scale images using weber local binary image based cosine transform features. Multimed Tools Appl 77(1):1133–1165
Kumar S, Kumari R, Sharma VK (2015) Fitness based position update in spider monkey optimization algorithm. Procedia Computer Science 62:442–449
Li M, Xu H, Huang X, Song Z, Liu X, Li X (2018) Facial expression recognition with identity and emotion joint learning. IEEE Trans Affect Comput:1–1
Liang D, Liang H, Yu Z, Zhang Y (2020) Deep convolutional BiLSTM fusion network for facial expression recognition. Vis Comput 36(3):499–508
Liu Y, Zeng J, Shan S and Zheng Z (2018) Multi-Channel pose-aware convolution neural networks for multi-view facial expression recognition, in Automatic Face & Gesture Recognition (FG 2018), 2018 13th IEEE international conference on IEEE, 458-465
Lopes AT, de Aguiar E, De Souza AF, Oliveira-Santos T (2017) Facial expression recognition with convolutional neural networks: coping with few data and the training sample order. Pattern Recogn 61:610–628
Luo Y, Wu C, Zhang Y (2013) Facial expression recognition based on fusion feature of PCA and LBP with SVM. Optik - International Journal for Light and Electron Optics 124(17):2767–2770
Maheswari VU, Raju SV and Reddy KS (2019) Local directional weighted threshold patterns (LDWTP) for facial expression recognition. In 2019 fifth international conference on image information processing (ICIIP), IEEE, 167-170
Mandal M, Verma M, Mathur S, Vipparthi SK, Murala S, Kumar DK (2019) Regional adaptive affinitive patterns (RADAP) with logical operators for facial expression recognition. IET Image Process 13(5):850–861
Mavadati M, Sanger P and Mahoor MH (2016) Extended disfa dataset: investigating posed and spontaneous facial expressions. In proceedings of the IEEE conference on computer vision and pattern recognition workshops, 1-8
Muhammad G, Hussain M, Alenezy F, Bebis G, Mirza AM and Aboalsamh H (2012) Race recognition from face images using weber local descriptor. In systems, signals and image processing (IWSSIP), 2012 19th international conference on IEEE, 421-424
Munir A, Hussain A, Khan SA, Nadeem M and Arshid S (2018) Illumination-invariant facial expression recognition using selected merged binary patterns for real world images, Optik-International Journal for Light and Electron Optics, Illumination invariant facial expression recognition using selected merged binary patterns for real world images
Naik S and Jagannath R (2018) GCV-based regularized extreme learning machine for facial expression recognition, advances in intelligent systems and computing, 129-138
Ryu B, Rivera AR, Kim J, Chae O (2017) Local directional ternary pattern for facial expression recognition. IEEE Trans Image Process 26(12):6006–6018
Sajjad M, Shah A, Jan Z, Shah SI, Baik SW and Mehmood I (2017) Facial appearance and texture feature-based robust facial expression recognition framework for sentiment knowledge discovery, Cluster Computing, 1-19
Sajjad M, Zahir S, Ullah A, Akhtar Z and Muhammad K (2019) Human behavior understanding in big multimedia data using CNN based facial expression recognition. Mobile networks and applications, 1-11
Sarode N, Bhatia S (2010) Facial expression recognition. International Journal on computer science and Engineering 2(5):1552–1557
Shah N and Priyanka (2018) An Improved Framework for Human Face Recognition, Advances in Intelligent Systems and Computing, 175–180
Sharma R, Patterh MS (2015) A new pose-invariant face recognition system using PCA and ANFIS. Optik-International Journal for Light and Electron Optics 126(23):3483–3487
Sun Z, Hu Z-P, Chiong R, Wang M and Zhao S (2018) An adaptive weighted fusion model with two subspaces for facial expression recognition, Signal, Image, and Video Processing, 1–9
Sun Z, Hu Z-p, Wang M, Zhao S-H (2019) Dictionary learning feature space via sparse representation classification for facial expression recognition. Artif Intell Rev 51(1):1–18
Tripathi A, Pandey S and Jangir H (2018) Efficient Facial Expression Recognition System Based on Geometric Features Using Neural Network, In Information and Communication Technology for Sustainable Development, Springer, Singapore 181–190.
Uddin MZ, Hassan MM, Almogren A, Alamri A, Alrubaian M, Fortino G (2017) Facial expression recognition utilizing local direction-based robust features and deep belief network. IEEE Access 5:4525–4536
Uddin MZ, Hassan MM, Almogren A, Zuair M, Fortino G, Torresen J (2017) A facial expression recognition system using robust face features from depth videos and deep learning. Computers & Electrical Engineering 63:114–125
Valstar MF, Sánchez-Lozano E, Cohn JF, Jeni LA, Girard JM, Zhang Z, Yin L and Fera PM (2017) 2017-addressing head pose in the third facial expression recognition and analysis challenge, in Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on IEEE, 839-847
Verma M, Vipparthi SK, Singh G (2019) Hinet: hybrid inherited feature learning network for facial expression recognition. IEEE Letters of the Computer Society 2(4):36–39
Videla SL, Rao MRN, Anand D, Vankayalapati HD and Razia S (2019) Deformable facial fitting using active appearance model for emotion recognition. In Smart Intelligent Computing and Applications, Springer, Singapore 135–144
Xie W, Jia X, Shen L, Yang M (2019) Sparse deep feature learning for facial expression recognition. Pattern Recogn 96:106966
Xu, Lei, Minrui Fei, Wenju Zhou, and Aolei Yang (2008) Face expression recognition based on convolutional neural network. In 2018 Australian & New Zealand Control Conference (ANZCC), pp. 115–118. IEEE
Yang J and Wang S (2017) Capturing spatial and temporal patterns for distinguishing between posed and spontaneous expressions. In proceedings of the 2017 ACM on multimedia conference, 469-477
Yang R-P, Liu Z-T, Zheng L-D, Wu J-P and C-C Hu (2018) Intelligent Mirror System Based on Facial Expression Recognition and Color Emotion Adaptation iMirror. In 2018 37th Chinese Control Conference (CCC), IEEE, 3227–3232.
Yang Y, Fang D, Zhu D (2016) Facial expression recognition using deep belief network. Rev Tec Ing Univ Zulia 39(2):384–392
Yuan B, Cao H and Chu J (2012) Combining local binary pattern and local phase quantization for face recognition. In 2012 international symposium on biometrics and security technologies, IEEE 51-53
Zeng N, Zhang H, Song B, Liu W, Li Y, Dobaie A (2018) Facial expression recognition via learning deep sparse autoencoders. Neurocomputing 273:643–649
Zhang K, Huang Y, Du Y (2017) Facial expression recognition based on deep evolutional spatial-temporal networks. IEEE transactions on image processing, 4193-4203
Zhang T, Zheng W, Cui Z, Zong Y, Yan J, Yan K (2016) A deep neural network-driven feature learning method for multi-view facial expression recognition. IEEE Transactions on Multimedia 18(12):2528–2536
Zhang W, Zhao X, Morvan J and Chen L (2018) Improving shadow suppression for illumination robust face recognition. IEEE Trans Pattern Anal Mach Intell, 1–1
Zhang Z, Luo P, Loy CC, Tang X (2018) From facial expression recognition to interpersonal relation prediction. Int J Comput Vis 126(5):550–569
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Vedantham, R., Reddy, E.S. A robust feature extraction with optimized DBN-SMO for facial expression recognition. Multimed Tools Appl 79, 21487–21512 (2020). https://doi.org/10.1007/s11042-020-08901-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-08901-x