Skip to main content
SpringerLink
Log in

Novel high-dimensional phase space features for EEG emotion recognition

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Currently, a fundamental role of emotion recognition is apparent in both medical and non-medical applications. The current work envisioned providing novel procedures to characterize electroencephalography (EEG) dynamics during the exposure of emotional provocations. These features were basically obtained from a high-dimensional phase space. Furthermore, we examined if the number of selected features influences emotion recognition rates. We assessed the emotion recognition rates of the proposed system using Naïve Bayes and the k-nearest neighbor (kNN) using the k-fold cross-validation (CV) strategy. A 62-channel EEG data of 15 volunteers available at the SJTU Emotion EEG Dataset-IV (SEED-IV) were examined, while participants were watching sad, happy, fearful, and neutral videos. Our results showed that (1) by performing the statistical test, the highest number of significant differences were found between fear and sadness; (2) using a 12-fold CV and selecting nine top-ranked features, 6-NN outperformed the other kNN classification schemes. In this case, the highest accuracy rate of 88.89% was achieved; (3) Naïve Bayes achieved the highest performance of 100%, in which the number of selected features was five. In conclusion, the effectiveness of the proposed novel measures was shown for recognizing EEG-affective states.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Al-Nafjan, A., Hosny, M., Al-Ohali, Y., Al-Wabil, A.: Review and classification of emotion recognition based on EEG Brain-computer interface system research: a systematic review. Appl. Sci. 7(12), 1239 (2017)

    Article  Google Scholar 

  2. Sheng, W., Li, X.: Multi-task learning for gait-based identity recognition and emotion recognition using attention enhanced temporal graph convolutional network. Pattern Recognit. 114, 107868 (2021)

    Article  Google Scholar 

  3. Wang, X., Chen, X., Cao, C.: Human emotion recognition by optimally fusing facial expression and speech feature. Signal Process. Image Commun. 84, 115831 (2020)

    Article  Google Scholar 

  4. Maffei, A., Angrilli, A.: Spontaneous blink rate as an index of attention and emotion during film clips viewing. Physiol. Behav. 204, 256–263 (2019)

    Article  Google Scholar 

  5. Goshvarpour, A., Goshvarpour, A.: Evaluation of novel entropy-based complex wavelet sub-bands measures of PPG in an emotion recognition system. J. Med. Biol. Eng. 40, 451–461 (2020)

    Article  Google Scholar 

  6. Goshvarpour, A., Goshvarpour, A.: The potential of photoplethysmogram and galvanic skin response in emotion recognition using nonlinear features. Phys. Eng. Sci. Med. 43, 119–134 (2020)

    Article  Google Scholar 

  7. Goshvarpour, A., Goshvarpour, A.: Poincaré’s section analysis for PPG-based automatic emotion recognition. Chaos Solitons Fractals 114, 400–407 (2018)

    Article  Google Scholar 

  8. Goshvarpour, A., Goshvarpour, A.: A novel feature level fusion for HRV classification using correntropy and Cauchy–Schwarz divergence. J. Med. Syst. 42, 109 (2018)

    Article  Google Scholar 

  9. Goshvarpour, A., Goshvarpour, A., Abbasi, A.: Evaluation of signal processing techniques in discriminating ECG signals of men and women during rest condition and emotional states. Biomed. Eng. Appl. Basis Commun. 30(4), 1850028 (2018)

    Article  Google Scholar 

  10. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: An accurate emotion recognition system using ECG and GSR signals and matching pursuit method. Biomed. J. 40, 355–368 (2017)

    Article  Google Scholar 

  11. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Fusion of heart rate variability and pulse rate variability for emotion recognition using lagged poincare plots. Aust. Phys. Eng. Sci. Med. 40(3), 617–629 (2017)

    Article  Google Scholar 

  12. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Do men and women have different ECG responses to sad pictures? Biomed. Signal Process. Control 38, 67–73 (2017)

    Article  Google Scholar 

  13. Goshvarpour, A., Abbasi, A., Goshvarpour, A., Daneshvar, S.: Discrimination between different emotional states based on the chaotic behavior of galvanic skin responses. SIViP 11(7), 1347–1355 (2017)

    Article  Google Scholar 

  14. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Indices from lagged poincare plots of heart rate variability: an efficient nonlinear tool for emotion discrimination. Aust. Phys. Eng. Sci. Med. 40(2), 277–287 (2017)

    Article  Google Scholar 

  15. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Multi-aspects of emotional electrocardiogram classification in combination with musical stimuli and composite features. Int. J. Appl. Pattern Recognit. 4(1), 64–88 (2017)

    Article  Google Scholar 

  16. Goshvarpour, A., Abbasi, A., Goshvarpour, A., Daneshvar, S.: A novel signal-based fusion approach for accurate music emotion recognition. Biomed. Eng. Appl. Basis Commun. 28(6), 1650040 (2016)

    Article  Google Scholar 

  17. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Gender differences in response to affective audio and visual inductions: examination of nonlinear dynamics of autonomic signals. Biomed. Eng. Appl. Basis Commun. 28(4), 1650024 (2016)

    Article  Google Scholar 

  18. Goshvarpour, A., Abbasi, A., Goshvarpour, A., Daneshvar, S.: Fusion framework for emotional ECG and GSR recognition applying wavelet transform. Iran. J. Med. Phys. 13(3), 163–173 (2016)

    Google Scholar 

  19. Bigdely-Shamlo, N., Mullen, T., Kothe, C., Su, K.-M., Robbins, K.A.: The PREP pipeline: standardized preprocessing for large-scale EEG analysis. Front. Neuroinform. 9, 16 (2015)

    Article  Google Scholar 

  20. Tuncer, T., Dogan, S., Subasi, A.: A new fractal pattern feature generation function based emotion recognition method using EEG. Chaos Solitons Fractals 144, 110671 (2021)

    Article  MathSciNet  Google Scholar 

  21. Nawaz, R., Hwa Cheah, K., Nisar, H., Yap, V.V.: Comparison of different feature extraction methods for EEG-based emotion recognition. Biocybern. Biomed. Eng. 40(3), 910–926 (2020)

    Article  Google Scholar 

  22. Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7, 162–175 (2015)

    Article  Google Scholar 

  23. Lan, Z., Sourina, O., Wang, L., et al.: Real-time EEG-based emotion monitoring using stable features. Vis. Comput 32, 347–358 (2016)

    Article  Google Scholar 

  24. Goshvarpour, A., Goshvarpour, A.: EEG spectral powers and source localization in depressing, sad, and fun music videos focusing on gender differences. Cogn. Neurodyn. 13(2), 161–173 (2019)

    Article  Google Scholar 

  25. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Dynamical analysis of emotional states from electroencephalogram signals. Biomed. Eng. Appl. Basis Commun. 28(2), 1650015 (2016)

    Article  Google Scholar 

  26. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Combination of sLORETA and nonlinear coupling for emotional EEG source localization. Nonlinear Dyn. Psychol. Life Sci. 20(3), 353–368 (2016)

    Google Scholar 

  27. Goshvarpour, A., Abbasi, A., Goshvarpour, A.: Recurrence quantification analysis and neural networks for emotional EEG classification. Appl. Med. Inform 38(1), 13–24 (2016)

    Google Scholar 

  28. Alotaiby, T., El-Samie, F.E.A., Alshebeili, S.A., et al.: A review of channel selection algorithms for EEG signal processing. EURASIP J. Adv. Signal Process. 2015, 66 (2015)

    Article  Google Scholar 

  29. Goshvarpour, A., Goshvarpour, A.: A novel approach for EEG electrode selection in automated emotion recognition based on lagged Poincare’s indices and sLORETA. Cogn. Comput. 12, 602–618 (2020)

    Article  Google Scholar 

  30. Salankar, N., Mishra, P., Garg, L.: Emotion recognition from EEG signals using empirical mode decomposition and second-order difference plot. Biomed. Signal Process. Control 65, 102389 (2021)

    Article  Google Scholar 

  31. Hou, H.R., Zhang, X.N., Meng, Q.H.: Odor-induced emotion recognition based on average frequency band division of EEG signals. J. Neurosci. Methods 334, 108599 (2020)

    Article  Google Scholar 

  32. Li, H., Jin, Y.-M., Zheng, W.-L., Lu, B.-L.: Cross-subject emotion recognition using deep adaptation networks. In: International Conference on Neural Information Processing, Springer, pp. 403–413 (2018)

  33. Li, Y., Zheng, W., Zong, Y., Cui, Z., Zhang, T., Zhou, X.: A bi-hemisphere domain adversarial neural network model for EEG emotion recognition. IEEE Trans. Affect. Comput. 12(2), 494–504 (2018)

    Article  Google Scholar 

  34. Song, T., Zheng, W., Song, P., Cui, Z.: EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans. Affect. Comput. 11, 532–541 (2019)

    Article  Google Scholar 

  35. Zhong, P., Wang, D., Miao, C.: EEG-based emotion recognition using regularized graph neural networks. IEEE Trans. Affect. Comput. (2020). https://doi.org/10.1109/TAFFC.2020.2994159

    Article  Google Scholar 

  36. Li, Y., Wang, L., Zheng, W., Zong, Y., Qi, L., Cui, Z., et al.: A novel Bi-hemispheric discrepancy model for EEG emotion recognition. IEEE Trans. Cognit. Dev. Syst. 13(2), 354–367 (2021)

    Article  Google Scholar 

  37. Li, J., Li, S., Pan, J., Wang, F.: Cross-subject EEG emotion recognition with self-organized graph neural network. Front. Neurosci. 15, 611653 (2021)

    Article  Google Scholar 

  38. Yin, Y., Zheng, X., Hu, B., Zhang, Y., Cui, X.: EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM. Appl. Soft Comput. 100, 106954 (2021)

    Article  Google Scholar 

  39. Hsu, Y.-L., Wang, J.-S., Chiang, W.-C., Hung, C.-H.: Automatic ECG-based emotion recognition in music listening. IEEE Trans. Affect. Comput. 11(1), 85–99 (2020)

    Article  Google Scholar 

  40. Yang, S.: Nonlinear signal classification using geometric statistical features in state space. Electron. Lett. 40, 780–781 (2004)

    Article  Google Scholar 

  41. Yang, S.: Nonlinear signal classification in the framework of high-dimensional shape analysis in reconstructed state space. IEEE Trans. Circuits Syst. II Express Briefs 52, 512–516 (2005)

    Article  Google Scholar 

  42. Zheng, W.L., Liu, W., Lu, Y., Lu, B.L., Cichocki, A.: Emotionmeter: a multimodal framework for recognizing human emotions. IEEE Trans. Cybern. 49(3), 1110–1122 (2019)

    Article  Google Scholar 

  43. Fraser, A., Swinney, H.: Independent coordinates for strange attractors from mutual information. Phys. Rev. A Gen. Phys. 33(2), 1134–1140 (1986)

    Article  MathSciNet  MATH  Google Scholar 

  44. Wallot, S., Mønster, D.: Calculation of average mutual information (AMI) and false-nearest neighbors (FNN) for the estimation of embedding parameters of multidimensional time series in matlab. Front. Psychol. 9, 1679 (2018)

    Article  Google Scholar 

  45. Hotelling, H.: Analysis of a complex of statistical variables into principal components. J. Educ. Psychol. 24(6), 417–441 (1933)

    Article  MATH  Google Scholar 

  46. Zheng, W.L., Zhu, J.Y., Lu, B.L.: Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans. Affect. Comput. 10(3), 417–429 (2016)

    Article  Google Scholar 

  47. Wang, F., Zhong, S., Peng, J., Jiang, J., Liu, Y.: Data augmentation for EEG-based emotion recognition with deep convolutional neural networks. In: Schoeffmann, K. et al. (eds.), MultiMedia Modeling. MMM 2018. Lecture Notes in Computer Science, vol. 10705, Springer, Cham (2018)

  48. Thompson, N.C., Greenewald, K., Lee, K., Manso, G.F.: The Computational Limits of Deep Learning. https://arxiv.org/abs/2007.05558

  49. Soroush, M.Z., Maghooli, K., Setarehdan, S.K., Nasrabadi, A.M.: Emotion recognition through EEG phase space dynamics and Dempster-Shafer theory. Med. Hypoth. 127, 34–45 (2019)

    Article  Google Scholar 

Download references

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Faculty of Electrical Engineering, Sahand University of Technology, Tabriz, Iran

    Atefeh Goshvarpour

  2. Department of Biomedical Engineering, Imam Reza International University, Mashhad, Razavi Khorasan, Iran

    Ateke Goshvarpour

  3. Health Technology Research Center, Imam Reza International University, Mashhad, Razavi Khorasan, Iran

    Ateke Goshvarpour

Authors
  1. Atefeh Goshvarpour
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ateke Goshvarpour
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ateke Goshvarpour.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article examined EEG signals of the SEED-IV dataset [42], which is freely available in the public domain. This article does not contain any studies with human participants performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study [42].

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Goshvarpour, A., Goshvarpour, A. Novel high-dimensional phase space features for EEG emotion recognition. SIViP 17, 417–425 (2023). https://doi.org/10.1007/s11760-022-02248-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-022-02248-6

Keywords

Search

Navigation