Skip to main content
SpringerLink
Log in

EEG emotion recognition using multichannel weighted multiscale permutation entropy

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Electroencephalogram (EEG) signal is a time-varying and nonlinear spatial discrete signal, which has been widely used in the field of emotion recognition. Up to now, a large number of studies have chosen time–frequency domain features or extracted features through brain networks. However, partial spatial or time–frequency information of EEG signals will be lost when analyzing from a single point of view. At the same time, the network analysis based on EEG is largely affected by the inherent volume effect of EEG. Therefore, how to eliminate the influence of volume effect on brain network analysis and extract the features that can reflect both time–frequency information and spatial information is the problem we need to solve at present. In this paper, a feature fusion method that can better reflect the emotional state is proposed. This method uses multichannel weighted multiscale permutation entropy (MC-WMPE) as the feature. It not only takes into account the time–frequency and spatial information of EEG signals but also eliminates the inherent volume effect of EEG signals. We first calculate the multiscale permutation entropy (MPE) of the EEG signals in each channel and construct the brain functional network by calculating the Pearson correlation coefficient (PCC) between each channel. PageRank algorithm is used to sort the importance of nodes in the brain functional network, and the weight of each node is obtained to screen out the important channels in emotion recognition. Then the weights of each channel and the MPE are weighted combined to obtain MC-WMPE as the feature. The research shows that both temporal information and spatial information are of great significance in processing EEG signals. Moreover, the analysis of the frontal, parietal and occipital lobes is necessary for studying the activity state of the cerebral cortex under emotional stimulation. Finally, we carried out experiments on the DEAP and SEED database, and the highest accuracy rate of emotion recognition with this combination feature is 85.28% and 87.31%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Data availability

All code and data are available.

References

  1. Rotem-Kohavi N, Oberlander TF, Virji-Babul N (2017) Infants and adults have similar regional functional brain organization for the perception of emotions. Neuroence Letters 650:118–125. https://doi.org/10.1016/j.neulet.2017.04.031

    Article  Google Scholar 

  2. Yan J, Zheng W, Xin M et al (2014) Integrating facial expression and body gesture in videos for emotion recognition. IEICE Trans Inf Syst 97(3):610–613

    Article  Google Scholar 

  3. Khosrowabadi R, Quek C, Ang KK et al (2013) ERNN: A biologically inspired feedforward neural network to discriminate emotion from EEG signal. IEEE Transactions on Neural Networks and Learning Systems 25(3):609–620. https://doi.org/10.1109/TNNLS.2013.2280271

    Article  Google Scholar 

  4. Hamada M et al (2018) A systematic review for human EEG brain signals based emotion classification, feature extraction, brain condition, group comparison. J Med Syst 42(9):1–25

    Article  Google Scholar 

  5. Alarcao SM, Fonseca MJ (2017) Emotions recognition using EEG signals: a survey. IEEE Trans Affect Comput 10(3):374–393. https://doi.org/10.1109/TAFFC.2017.2714671

    Article  Google Scholar 

  6. Mahsa Vaghefi, Ali Motie Nasrabadi, et al. (2019) Nonlinear analysis of electroencephalogram signals while listening to the holy Quran. Journal of Medical Signals and Sensors, 9( 2):100–110

  7. Xiang J, Rui C, Li Li (2014) Emotion recognition based on the sample entropy of EEG. Bio-Med Mater Eng 24(1):1185–1192. https://doi.org/10.3233/BME-130919

    Article  Google Scholar 

  8. Kaur, A., Verma, K., Bhondekar, A.P., Shashvat, K (2020) Comparison of classification models using entropy based features from sub-bands of EEG. Traitement du Signal, Vol. 37, No. 2, pp. 279–289. https://doi.org/10.18280/ts.370214

  9. Shah, S.A.A., Habib, N., Nadeem, M.S.A., et al. (2020) Extraction of dynamical information and classification of heart rate variability signals using scale based permutation entropy measures. Traitement du Signal, Vol. 37, No. 3, pp. 355–365. https://doi.org/10.18280/ts.370302

  10. Aziz Wajid, Arif M (2005) Multiscale permutation entropy of physiological time series. 2005 Pakistan Section Multitopic Conference, Karachi, pp. 1–6. 10.1109 / INMIC.2005.334494

  11. Khoshnoud Shiva, Nazari Mohammad Ali, et al. (2018) Functional brain dynamic analysis of ADHD and control children using nonlinear dynamical features of EEG signals. vol.17, no. 1, pp. 17-30. https://doi.org/10.3233/JIN-170033

  12. Jenke R, Peer A, Buss M. Peer, et al. (2014) Feature extraction and selection for emotion recognition from EEG. IEEE transactions on affective computing, vol. 5, no. 3, pp. 327–339. 10.1109 / TAFFC.2014.2339834

  13. Peter, et al. Personality Profiles Are Associated with Functional Brain Networks Related to Cognition and Emotion. Scientific reports, vol. 8, no. 1, p. 13874. 2018. https://doi.org/10.1038/s41598-018-32248-x

  14. Piqueira JRC (2011) Network of phase-locking oscillators and a possible model for neural synchronization. Commun Nonlinear Sci Numer Simul 16(9):3844–3854. https://doi.org/10.1016/j.cnsns.2010.12.031

    Article  MathSciNet  MATH  Google Scholar 

  15. Bob P, Susta M et al (2009) Dissociative symptoms and interregional EEG cross-correlations in paranoid schizophrenia. Psychiatry Res 177(1–2):37–40. https://doi.org/10.1016/j.psychres.2009.08.015

    Article  Google Scholar 

  16. Dissanayaka, Ben-Simon, Gruberger, et al. Comparison between human awake, meditation and drowsiness EEG activities based on directed transfer function and MVDR coherence methods. Medical & Biological Engineering & Computing, vol. 53, pp. 599–607. 2015. https://doi.org/10.1007/s11517-015-1272-0

  17. Zare M, Rezvani Z et al (2016) Automatic classification of 6-month-old infants at familial risk for language-based learning disorder using a support vector machine. Clin Neurophysiol 127(7):2695–2703. https://doi.org/10.1016/j.clinph.2016.03.025

    Article  Google Scholar 

  18. Chandaka S, Chatterjee A, Munshi S (2009) Cross-correlation aided support vector machine classifier for classification of EEG signals. Expert Syst Appl 36(2):1329–1336. https://doi.org/10.1016/j.eswa.2007.11.017

    Article  Google Scholar 

  19. Shahabi H, Moghimi S (2016) Toward automatic detection of brain responses to emotional music through analysis of EEG effective connectivity. Comput Hum Behav 58:231–239. https://doi.org/10.1016/j.chb.2016.01.005

    Article  Google Scholar 

  20. Mishuhina V, Jiang X (2021) Complex common spatial patterns on time-frequency decomposed EEG for Brain-Computer Interface[J]. Pattern Recognition, vol. 115, no. 1, pp.107918, July. 2021. https://doi.org/10.1016/j.patcog.2021.107918

  21. Bhattacharyya A, Ranta R, Le Cam S et al (2018) A multi-channel approach for cortical stimulation artefact suppression in depth EEG signals using time-frequency and spatial filtering[J]. IEEE Trans Biomed Eng. https://doi.org/10.1109/TBME.2018.2881051

    Article  Google Scholar 

  22. Azami H, Escudero J (2016) Improved multiscale permutation entropy for biomedical signal analysis: Interpretation and application to electroencephalogram recordings. Biomed Signal Process Control 23:28–41. https://doi.org/10.1016/j.bspc.2015.08.004

    Article  Google Scholar 

  23. Wang Y, Jia Z, Zeng L (2018) Coarse Graining Method Based on Noded Similarity in Complex Network. Commun Netw 10(3):51–64. https://doi.org/10.4236/cn.2018.103005

    Article  Google Scholar 

  24. Liu T, Yao W, Min Wu et al (2017) Multiscale permutation entropy analysis of electrocardiogram. Physica A 471:492–498. https://doi.org/10.1016/j.physa.2016.11.102

    Article  Google Scholar 

  25. Christoph B, Bernd P (2002) Permutation entropy: a natural complexity measure for time series. Phys Rev Lett 88(17):174102. https://doi.org/10.1103/PhysRevLett.88.174102

    Article  Google Scholar 

  26. Zeng Ke, Gaoxiang, Chen He, et al. (2018) Characterizing dynamics of absence seizure EEG with spatial-temporal permutation entropy. Neurocomputing, vol. 275, pp. 577–585. https://doi.org/10.1016/j.neucom.2017.09.007

  27. Choi Y-S (2017) Improved multiscale permutation entropy measure for analysis of brain waves. International Journal of Fuzzy Logic and Intelligent System 17(3):194–201. https://doi.org/10.5391/IJFIS.2017.17.3.194

    Article  Google Scholar 

  28. Bhavsar R, Davey N, Helian Na et al (2018) Time series analysis using embedding dimension on heart rate variability. Procedia Computer Science 145:89–96. https://doi.org/10.1016/j.procs.2018.11.015

    Article  Google Scholar 

  29. Peiyang Li et al (2019) EEG based emotion recognition by combining functional connectivity network and local activations. IEEE Trans Biomed Eng 66(10):2869–2881. https://doi.org/10.1109/TBME.2019.2897651

    Article  Google Scholar 

  30. Bönstrup M, Schulz R et al (2016) Dynamic causal modelling of EEG and fMRI to characterize network architectures in a simple motor task. Neuroimage 124:498–508. https://doi.org/10.1016/j.neuroimage.2015.08.052

    Article  Google Scholar 

  31. Tóth B, Urbán G et al (2017) Large-scale network organization of EEG functional connectivity in newborn infants. Hum Brain Mapp 38(8):4019–4033. https://doi.org/10.1002/hbm.23645

    Article  Google Scholar 

  32. Giroldini W et al (2016) A new method to detect event-related potentials based on Pearson’s correlation. EURASIP J Bioinf Syst Biol 2016(1):11. https://doi.org/10.2139/ssrn.2609008

    Article  Google Scholar 

  33. Jian W, Xingshu C, Dengqi Y (2013) A study of important node rank based on KAD network. Adv Sci Lett 19(8):2266–2270. https://doi.org/10.1166/asl.2013.4963

    Article  Google Scholar 

  34. Chen G et al (2020) Nonnegative matrix factorization for link prediction in directed complex networks using PageRank and asymmetric link clustering information. Expert Syst Appl 148:113290. https://doi.org/10.1016/j.eswa.2020.113290

    Article  Google Scholar 

  35. Chen F, Zhang Y, Rohe K (2020) Targeted sampling from massive block model graphs with personalized PageRank. Journal of the Royal Statistical Society: Series B (Statistical Methodology) 82(1):99–126. https://doi.org/10.1111/rssb.12349

    Article  MathSciNet  MATH  Google Scholar 

  36. S. Koelstra, C. Muhl, M. Soleymani M, et al. (2012) DEAP: A database for emotion analysis; using physiological signals. IEEE Transactions on Affective Computing, vol. 3, no. 1, pp. 18–32. 2012. https://doi.org/10.1109/T-AFFC.2011.15

  37. Duan R N, Zhu J Y, Lu B L. (2013) Differential entropy feature for EEG-based emotion classification[C]//Neural Engineering (NER), 2013 6th International IEEE/EMBS Conference on. IEEE, pp. 81–84

  38. Zheng WL, Lu BL (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks[J]. IEEE Trans Auton Ment Dev 7(3):162–175. https://doi.org/10.1109/TAMD.2015.2431497

    Article  Google Scholar 

  39. V Pranava Bhargavi, Rajasekhar Bandara, et al. (2018) Classification of a MRI brain image using genetic algorithm for KNN classifier. Indian Journal of Public Health Research & Development, vol. 9, no. 10, pp. 1031–1033. https://doi.org/10.5958/0976-5506.2018.01267.6

  40. Wang Z, Yang C et al (2018) Multi-radial basis function SVM classifier: design and analysis. J Electric Eng Technol 13(6):2511–2520. https://doi.org/10.5370/JEET.2018.13.6.2511

    Article  Google Scholar 

  41. Manju N, Harish BS et al (2019) Ensemble feature selection and classification of internet traffic using xgboost classifier. Int J Comput Netw Inform Secur 11(7):37–44. https://doi.org/10.5815/ijcnis.2019.07.06

    Article  Google Scholar 

  42. M. K. Ahirwal and M. R. Kose (2018) Emotion Recognition System based on EEG signal: A comparative study of different features and classifiers. 2018 Second International Conference on Computing Methodologies and Communication, pp. 472–476

  43. Fatemeh B et al (2013) EEG-based emotion recognition using Recurrence Plot analysis and K nearest neighbor classifier. Biomedical Engineering IEEE 18:228–233. https://doi.org/10.1109/ICBME.2013.6782224

    Article  Google Scholar 

  44. Ouyang G et al (2013) Dynamic characteristics of absence EEG recordings with multiscale permutation entropy analysis. Epilepsy Res 104(3):246–252. https://doi.org/10.1016/j.eplepsyres.2012.11.003

    Article  Google Scholar 

  45. Liu X, Bin Hu, Zheng X et al (2019) Facial expression awareness based on multi-scale permutation entropy of EEG[J]. Int J Data Min Bioinform 21(4):287–300. https://doi.org/10.1504/IJDMB.2018.098936

    Article  Google Scholar 

  46. Asghar MA, Khan MJ, Rizwan M et al (2021) AI inspired EEG-based spatial feature selection method using multivariate empirical mode decomposition for emotion classification[J]. Multimedia Syst. https://doi.org/10.1007/s00530-021-00782-w

    Article  Google Scholar 

  47. Torres PE, Martins GH, Ribeiro VL et al (2021) Empirical evidence relating EEG signal duration to emotion classification performance[J]. IEEE Trans Affect Comput 12(1):154–164. https://doi.org/10.1109/TAFFC.2018.2854168

    Article  Google Scholar 

  48. Qinghua Z, Yongsheng Z, Dongli C et al (2020) Electroencephalogram access for emotion recognition based on a deep hybrid network[J]. Front Hum Neurosci. https://doi.org/10.3389/fnhum.2020.589001

    Article  Google Scholar 

  49. Lew W C L, Wang D, Shylouskaya K, et al. (2020) EEG-based emotion recognition using spatial-temporal representation via Bi-GRU[C]. 2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), pp. 116–119https://doi.org/10.1109/EMBC44109.2020.9176682

  50. Wang Z, Tong Y, Heng X (2019) Phase-locking value based graph convolutional neural networks for emotion recognition[J]. IEEE Access 7:93711–93722. https://doi.org/10.1109/ACCESS.2019.2927768

    Article  Google Scholar 

  51. Yang S, Wang J, Deng B, et al. (2021) Neuromorphic context-dependent learning framework with fault-tolerant spike routing[J]. IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15. https://doi.org/10.1109/TNNLS.2021.3084250

  52. Yang S, J Wang, N Zhang, et al. (2021) CerebelluMorphic: Large-Scale Neuromorphic Model and Architecture for Supervised Motor Learning[J]. IEEE Transactions on Neural Networks and Learning Systems, pp. 1–15. https://doi.org/10.1109/TNNLS.2021.3057070

Download references

Funding

This work was supported in part by the National Natural Science Foundation of China under Grant 61373116, in part by the National Natural Science Foundation of China under Grant 62002287.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Xi’an University of Posts and Telecommunications, Xi’an, 710121, China

    Zhong-Min Wang, Jia-Wen Zhang, Yan He & Jie Zhang

  2. Shaanxi Key Laboratory of Network Data Analysis and Intelligent Processing, Xi’an University of Posts and Telecommunications, 710121, Xi’an, China

    Zhong-Min Wang, Yan He & Jie Zhang

Authors
  1. Zhong-Min Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia-Wen Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

ZhongMin Wang and JiaWen Zhang developed the idea of the study, participated in its design and coordination and helped to draft the manuscript. Yan He and Jie Zhang contributed to the acquisition and interpretation of data.

Corresponding authors

Correspondence to Zhong-Min Wang or Jia-Wen Zhang.

Ethics declarations

Ethics approval

All authors approve.

Consent to participate

All authors agree.

Consent for publication

All authors read and approved the final manuscript.

Conflicts of interest/Competing interests

No conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, ZM., Zhang, JW., He, Y. et al. EEG emotion recognition using multichannel weighted multiscale permutation entropy. Appl Intell 52, 12064–12076 (2022). https://doi.org/10.1007/s10489-021-03070-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03070-2

Keywords

Search

Navigation