Skip to main content

Advertisement

SpringerLink
Log in

EEG based emotion recognition using fusion feature extraction method

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

As a high-level function of the human brain, emotion is the external manifestation of people’s psychological characteristics. The emotion has a great impact on people’s personality and mental health. At the same time, emotion classification from electroencephalogram (EEG) signals have attracted much attention. To improve the precision of EEG-based emotion recognition, we proposed a fused feature extraction method to complete the classification of three emotions (neutral, happiness, and sadness). The standardized movie clips were selected to induce the corresponding emotion and the EEG response of 10 participants is collected by Emotiv EPOC. This paper systematically compared two kinds of EEG features (power spectrum and wavelet energy entropy) and their fusion for emotion classification. To reduce the dimension of fused features, we used principal component analysis (PCA) for dimensionality reduction and feature selection. The support vector machine (SVM) classifier and the relevance vector machines (RVM) classifier were utilized for emotion recognition respectively. From experimental results, we found that the fusion of two kinds of features outperformed a single feature for emotion classification by both the SVM classifier and the RVM classifier, and the averaged classification accuracy was 89.17% and 91.18%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Adeel A, Khan MA, Akram T, Sharif A, Yasmin M, Saba T, Javed K (2020) Entropy controlled deep features selection framework for grape leaf diseases recognition. Expert Syst

  2. Ahern GL, Schwartz GE (1985) Differential lateralization for positive and negative emotion in the human brain: EEG spectral analysis. Neuropsychologia 23:745–755

    Article  Google Scholar 

  3. Alarcao SM, Fonseca MJ (2017) Emotions recognition using EEG signals: a survey. IEEE Trans Affect Comput

  4. Alm C, Roth D, Sproat R (2005) Emotions from text: machine learning for text-based emotion prediction. In: Proceedings of the conference on human language technology and empirical methods in natural language processing, pp 579–586

  5. Arshad H, Khan MA, Sharif MI, Yasmin M, Tavares JMRS, Zhang Y, Satapathy SC (2020) A multilevel paradigm for deep convolutional neural network features selection with an application to human gait recognition. Expert Syst

  6. Candra H, Yuwono M, Chai R et al (2015) Investigation of window size in classification of EEG-emotion signal with wavelet entropy and support vector machine. In: The 37th annual international conference of the IEEE engineering in medicine and biology society

  7. Chang Chih-Chung, Lin Chih-Jen (2011) LIBSVM: a library for support vector machines

  8. de Cheveigné A, Di Liberto GM, Arzounian D et al (2018) Multiway canonical correlation analysis of brain data. NeuroImage

  9. Douglas-Cowie E, Cowie R, Schroder M (2000) A new emotion database: considerations, sources and scope. In: Proceedings of the ISCA workshop on Speech and emotion, pp 39–44

  10. Ekman P (1970) Universal facial expressions of emotion. California Mental Health Research Digest 8:151–158

    Google Scholar 

  11. Fernando L, Alonso N, Gomez-Gil J (2012) Brain computer interfaces, a review. Sensors 12:1211–1279

    Article  Google Scholar 

  12. Forster J, Strack F (1996) Influence of overt head movements on memory for valenced words: a case of conceptual–motor compatibility. J Pers Soc Psychol 71:421–430

    Article  Google Scholar 

  13. Geng X-S, She Q-S (2017) Classification of motion imagination EEG signals based on artificial bee Colony optimized gauss process. Chinese Journal of Sensors and Actuators 30:378–384

    Google Scholar 

  14. Hillman C, Rosengren K, Smith D (2004) Emotion and motivated behavior: postural adjustments to affective picture viewing. Biol Psychol 66:51–62

    Article  Google Scholar 

  15. Hubert W, de Jong-Meyer R (1991) Autonomic, neuroendocrine, and subjective responses to emotion-inducing film stimuli. Int J Psychophysiol 11:131–140

    Article  Google Scholar 

  16. Hussain N, Khan MA, Sharif M et al (2020) A deep neural network and classical features based scheme for objects recognition: an application for machine inspection. Multimedia Tools and Applications:1–23

  17. Hyvärinen A, Ramkumar P, Parkkonen L, Hari R (2010) Independent component analysis of short-time Fourier transforms for spontaneous EEG/MEG analysis. Neuroimage 49:257–271

    Article  Google Scholar 

  18. Iacoviello D, Petracca A, Spezialetti M, Placidi G (2015) A real-time classification algorithm for EEG-based BCI driven by self-induced emotions. Comput Methods Prog Biomed 122:293–303

    Article  Google Scholar 

  19. Ji N, Ma L, Dong H, Zhang X (2019) EEG signals feature extraction based on DWT and EMD combined with approximate entropy. Brain Sciences 9:201

    Article  Google Scholar 

  20. Jung TP, Makeig S, Stensmo M et al (1997) Estimating alertness from the EEG power spectrum. IEEE Trans Biomed Eng 44:60–69

    Article  Google Scholar 

  21. Khan M, Khan MA, Ahmed F, Mittal M, Goyal LM, Hemanth DJ, Satapathy SC (2019) Gastrointestinal diseases segmentation and classification based on duo-deep architectures. Pattern Recogn Lett

  22. Khosrowabadi R, Quek C, Ang KK et al (2014) ERNN: a biologically inspired feedforward neural network to discriminate emotion from EEG signal. IEEE Trans Affect Comput 25:609–620

    Google Scholar 

  23. Koelstra S, Muhl C, Soleymani M, Jong-Seok Lee, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis using physiological signals. IEEE Trans Affect Comput 3:18–31

    Article  Google Scholar 

  24. Lang PJ (1995) The emotion probe. Studies of motivation and attention. AmPsychol 50:372–385

    Google Scholar 

  25. Li D, Wang Z, GAO Q, Song Y, Yu X, Wang C (2019) Facial expression recognition based on electroencephalogram and facial landmark localization. Technol Health Care 27:373–387

    Article  Google Scholar 

  26. Liu YJ, Yu M, Ge Y et al (2018) Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Trans Affect Comput 9:550–562

    Article  Google Scholar 

  27. Majid A, Khan MA, Yasmin M, Rehman A, Yousafzai A, Tariq U (2020) Classification of stomach infections: a paradigm of convolutional neural network along with classical features fusion and selection. Microsc Res Tech 83:562–576

    Article  Google Scholar 

  28. Martis RJ, Acharya UR, Min LC (2013) ECG beat classification using PCA, LDA, ICA and discrete wavelet transform. Biomedical Signal Processing and Control 8:437–448

    Article  Google Scholar 

  29. Mehmood A, Khan MA, Sharif M, Khan SA, Shaheen M, Saba T, Ashraf I (2020) Prosperous human gait recognition: an end-to-end system based on pre-trained CNN features selection. Multimed Tools Appl

  30. Momennezhad A (2018) EEG-based emotion recognition utilizing wavelet coefficients. Multimed Tools Appl 77:27089–27106

    Article  Google Scholar 

  31. Morris JD (1995) Observations: SAM: the self-assessment manikin; an efficient cross-cultural measurement of emotional response

  32. Newson JJ, Thiagarajan TC (2019) EEG frequency bands in psychiatric disorders: a review of resting state studies. Front Hum Neurosci 12

  33. Pane ES, Wibawa AD, Purnomo MH (2019) Improving the accuracy of EEG emotion recognition by combining valence lateralization and ensemble learning with tuning parameters. Cogn Process 20:405–417

    Article  Google Scholar 

  34. Peng Y, Wang S, Long XZ et al (2015) Discriminative graph regularized extreme learning machine and its application to face recognition. Neurocomputing 49:340–353

    Article  Google Scholar 

  35. Perales FJ, Ramis S, Guerrero A et al (2019) Evaluation of a VR system for pain management using binaural acoustic stimulation. Multimed Tools Appl 78:32869–32890

    Article  Google Scholar 

  36. Rusell JA (1980) A circumplex model of affect. J Pers SocPsychol 39:116–121 178

    Article  Google Scholar 

  37. Scherer K (2004) Which emotions can be induced by music? what are the underlying mechanisms? and how can we measure them? J New Music Res 33:239–251

    Article  Google Scholar 

  38. Tao R, Li YL, Wang Y (2010) Short-time fractional Fourier transform and its applications. IEEE Trans Signal Process 58:2568–2580

    Article  MathSciNet  Google Scholar 

  39. Wang XW, Nie D, Lu BL (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing. 129:94–106

    Article  Google Scholar 

  40. Xue W, Xie Z, Liu C (2009) A soft sensor for the surface cleanliness level of ultra-smooth optical component based on the Genetic-algorithm and the Related Vector Machine. In: 2009 8th IEEE International Conference on Cognitive Informatics

  41. Zheng WL, Lu BL (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7:162–175

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the Fundamental Research on Advanced Technology and Engineering Application Team, Tianjin, China (grant No.20160524) and Natural Science Foundation of Tianjin (No.18JCYBJC87700).

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory for Control Theory and Application in Complicated Systems, Tianjin, China

    Qiang Gao, Chu-han Wang, Zhe Wang, En-zeng Dong & Yu Song

  2. Training Center of Tianjin University of Technology, Tianjin, China

    Xiao-lin Song

Authors
  1. Qiang Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chu-han Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhe Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiao-lin Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. En-zeng Dong
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yu Song
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yu Song.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, Q., Wang, Ch., Wang, Z. et al. EEG based emotion recognition using fusion feature extraction method. Multimed Tools Appl 79, 27057–27074 (2020). https://doi.org/10.1007/s11042-020-09354-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09354-y

Keywords

Search

Navigation