Skip to main content

Advertisement

SpringerLink
Log in

Accurate Emotion Recognition Utilizing Extracted EEG Sources as Graph Neural Network Nodes

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Automated analysis and recognition of human emotion play an important role in the development of a human–computer interface. High temporal resolution of EEG signals enables us to noninvasively study the emotional brain activities. However, one major obstacle in this procedure is extracting the essential information in presence of the low spatial resolution of EEG recordings. The pattern of each emotion is clearly defined by mapping from scalp sensors to brain sources using the standardized low-resolution electromagnetic tomography (sLORETA) method. A graph neural network (GNN) is then used for EEG-based emotion recognition in which sLORETA sources are considered as the nodes of the underlying graph. In the proposed method, the inter-source relations in EEG source signals are encoded in the adjacency matrix of GNN. Finally, the labels of the unseen emotions are recognized using a GNN classifier. The experiments on the recorded EEG dataset by inducing excitement through music (recorded in brain-computer interface research lab, University of Tabriz) indicate that the brain source activity modeling by ESB-G3N significantly improves the accuracy of emotion recognition. Experimental results show classification accuracy of 98.35% for two-class classification of positive and negative emotions. In this paper, we concentrate on extracting active emotional cortical sources using EEG source imaging (ESI) techniques. Auditory stimuli are used to rapidly and efficiently induce emotions in participants (visual stimuli in terms of video/image are either slow or inefficient in inducing emotions). We propose the use of active EEG sources as graph nodes by EEG source-based GNN node (ESB-G3N) algorithm. The results show an absolute improvement of 1–2% over subject-dependent and subject-independent scenarions compared to the existing approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data Availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Johnson WR. A study of the emotions of college athletes: Boston University; 1950.132 pages.

  2. Ngai WK, Xie H, Zou D, Chou KL. Emotion recognition based on convolutional neural networks and heterogeneous bio-signal data sources. Inf Fusion. 2022;77:107–17.

    Article  Google Scholar 

  3. Kreibig SD. Autonomic nervous system activity in emotion: a review. Biol Psychol. 2010;84(3):394–421.

    Article  Google Scholar 

  4. R. Williams D, Williams-Morris R. Racism and mental health: The African American experience. Ethn Health. 2000;5(3–4):243–68.

  5. Rotton J, Frey J. Air pollution, weather, and violent crimes: concomitant time-series analysis of archival data. J Pers Soc Psychol. 1985;49(5):1207.

    Article  Google Scholar 

  6. Sanei S. Adaptive processing of brain signals: John Wiley & Sons; 2013.

  7. Baillet S, Riera J, Marin G, Mangin J, Aubert J, Garnero L. Evaluation of inverse methods and head models for EEG source localization using a human skull phantom. Phys Med Biol. 2001;46(1):77–96.

    Article  Google Scholar 

  8. Hämäläinen M, Hari R, Ilmoniemi RJ, Knuutila J, Lounasmaa OV. Magnetoencephalography—theory, instrumentation, and applications to noninvasive studies of the working human brain. Rev Mod Phys. 1993;65(2):413.

    Article  Google Scholar 

  9. Michel CM, Murray MM. Towards the utilization of EEG as a brain imaging tool. Neuroimage. 2012;61(2):371–85.

    Article  Google Scholar 

  10. da Silva FL. EEG and MEG: relevance to neuroscience. Neuron. 2013;80(5):1112–28.

    Article  Google Scholar 

  11. Hu J, Tian J, Pan X, Liu J, editors. A comparison between EEG source localization and fMRI during the processing of emotional visual stimuli. Medical Imaging 2007: Physiology, Function, and Structure from Medical Images. Int Soc Opt Photonics. 2007.

  12. Batabyal T, Muthukrishnan S, Sharma R, Tayade P, Kaur S. Neural substrates of emotional interference: A quantitative EEG study. Neurosci Lett. 2018;685:1–6.

    Article  Google Scholar 

  13. Ekman P. Are there basic emotions? Psychol Rev. 1992; 99(3), 550–553.

  14. Tsolaki AC, Kosmidou VE, Kompatsiaris IY, Papadaniil C, Hadjileontiadis L, Tsolaki M. Age-induced differences in brain neural activation elicited by visual emotional stimuli: A high-density EEG study. Neuroscience. 2017;340:268–78.

    Article  Google Scholar 

  15. https://www.paulekman.com. 2004.

  16. Goshvarpour A, Goshvarpour A. EEG spectral powers and source localization in depressing, sad, and fun music videos focusing on gender differences. Cogn Neurodyn. 2019;13(2):161–73.

    Article  Google Scholar 

  17. Isotani T, Lehmann D, Pascual-Marqui RD, Fukushima M, Saito N, Yagyu T, et al. editors. Source localization of brain electric activity during positive, neutral and negative emotional states. International Congress Series. Elsevier. 2002.

  18. Pizzagalli D, Koenig T, Regard M, Lehmann D. Faces and emotions: brain electric field sources during covert emotional processing. Neuropsychologia. 1998;36(4):323–32.

    Article  Google Scholar 

  19. Vogel H, Szondi L. Lehrbuch der experimentellen Triebdiagnostik. Textband, 2. völlig umgearbeitete Auflage. Bern und Stuttgart Hans Hu. Psyche. 1960;14(8):860–1.

  20. Pourtois G, Delplanque S, Michel C, Vuilleumier P. Beyond conventional event-related brain potential (ERP): exploring the time-course of visual emotion processing using topographic and principal component analyses. Brain Topogr. 2008;20(4):265–77.

    Article  Google Scholar 

  21. Jäncke L, Alahmadi N. Detection of independent functional networks during music listening using electroencephalogram and sLORETA-ICA. NeuroReport. 2016;27(6):455–61.

    Article  Google Scholar 

  22. Padilla-Buritica JI, Martinez-Vargas JD, Castellanos-Dominguez G. Emotion discrimination using spatially compact regions of interest extracted from imaging EEG activity. Front Comput Neurosci. 2016;10:55.

    Article  Google Scholar 

  23. Chen G, Zhang X, Sun Y, Zhang J. Emotion feature analysis and recognition based on reconstructed eeg sources. IEEE Access. 2020;8:11907–16.

    Article  Google Scholar 

  24. Wang F, Wu S, Zhang W, Xu Z, Zhang Y, Wu C, et al. Emotion recognition with convolutional neural network and EEG-based EFDMs. Neuropsychologia. 2020;146:107506.

    Article  Google Scholar 

  25. https://bcmi.sjtu.edu.cn/. 2013.

  26. https://www.eecs.qmul.ac.uk/. 2012.

  27. Khare SK, Bajaj V. Time-frequency representation and convolutional neural network-based emotion recognition. IEEE Transactions on Neural Networks and Learning Systems. 2020.

  28. Wadhera T, Kakkar D, Rani R. Behavioral modeling using deep neural network framework for ASD diagnosis and prognosis. Emerging Technologies for Healthcare: Internet of Things and Deep Learning Models. 2021:279–98.

  29. Song T, Liu S, Zheng W, Zong Y, Cui Z, editors. Instance-adaptive graph for EEG emotion recognition. Proceedings of the AAAI Conference on Artificial Intelligence; 2020;34(03):2701-2708.

  30. Song T, Zheng W, Song P, Cui Z. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput. 2018;11(3):532–41.

    Article  Google Scholar 

  31. https://zenodo.org/record/546113. 2017.

  32. Zhong P, Wang D, Miao C. EEG-based emotion recognition using regularized graph neural networks. IEEE Transactions on Affective Computing. 2020.

  33. Jin L, Kim EY. Interpretable cross-subject EEG-based emotion recognition using channel-wise features. Sensors. 2020;20(23):6719.

    Article  Google Scholar 

  34. Plummer C, Harvey AS, Cook M. EEG source localization in focal epilepsy: where are we now? Epilepsia. 2008;49(2):201–18.

    Article  Google Scholar 

  35. Bauer H, Pllana A. EEG-based local brain activity feedback training—tomographic neurofeedback. Front Hum Neurosci. 2014;8:1005.

    Article  Google Scholar 

  36. Edelman BJ, Baxter B, He B. EEG source imaging enhances the decoding of complex right-hand motor imagery tasks. IEEE Trans Biomed Eng. 2015;63(1):4–14.

    Article  Google Scholar 

  37. Haufe S, Treder MS, Gugler MF, Sagebaum M, Curio G, Blankertz B. EEG potentials predict upcoming emergency brakings during simulated driving. J Neural Eng. 2011;8(5):056001.

    Article  Google Scholar 

  38. Noirhomme Q, Kitney RI, Macq B. Single-trial EEG source reconstruction for brain–computer interface. IEEE Trans Biomed Eng. 2008;55(5):1592–601.

    Article  Google Scholar 

  39. Grech R, Cassar T, Muscat J, Camilleri KP, Fabri SG, Zervakis M, et al. Review on solving the inverse problem in EEG source analysis. J Neuroeng Rehabil. 2008;5(1):1–33.

    Article  Google Scholar 

  40. Hallez H, Vanrumste B, Grech R, Muscat J, De Clercq W, Vergult A, et al. Review on solving the forward problem in EEG source analysis. J Neuroeng Rehabil. 2007;4(1):1–29.

    Article  Google Scholar 

  41. Hamalainen M. Interpreting measured magnetic fields of the brain: estimates of current distributions. Univ Helsinki, Finland Tech Rep TKK-F-A559. 1984.

  42. Pascual-Marqui RD, Michel CM, Lehmann D. Low resolution electromagnetic tomography: a new method for localizing electrical activity in the brain. Int J Psychophysiol. 1994;18(1):49–65.

    Article  Google Scholar 

  43. Pascual-Marqui RD. Standardized low-resolution brain electromagnetic tomography (sLORETA): technical details. Methods Find Exp Clin Pharmacol. 2002;24(Suppl D):5–12.

    Google Scholar 

  44. Kiebel SJ, Daunizeau J, Phillips C, Friston KJ. Variational Bayesian inversion of the equivalent current dipole model in EEG/MEG. Neuroimage. 2008;39(2):728–41.

    Article  Google Scholar 

  45. Mosher JC, Leahy RM, Lewis PS. EEG and MEG: forward solutions for inverse methods. IEEE Trans Biomed Eng. 1999;46(3):245–59.

    Article  Google Scholar 

  46. Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G. The graph neural network model. IEEE Trans Neural Networks. 2008;20(1):61–80.

    Article  Google Scholar 

  47. Meng Z, Adluru N, Kim HJ, Fung G, Singh V, editors. Efficient relative attribute learning using graph neural networks. Proceedings of the European conference on computer vision (ECCV). 2018.

  48. Goodfellow I, Bengio Y, Courville A. Deep learning, ser. The adaptive computation and machine learning series. Cambridge, MA: The MIT Press. 2016.

  49. https://bcilab.tabrizu.ac.ir/. 2017.

  50. Sheykhivand S, Mousavi Z, Rezaii TY, Farzamnia A. Recognizing emotions evoked by music using CNN-LSTM networks on EEG signals. IEEE Access. 2020;8:139332–45.

    Article  Google Scholar 

  51. Bradley MM, Lang PJ. Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry. 1994;25(1):49–59.

    Article  Google Scholar 

  52. Beck AT, Steer RA, Brown G. Beck depression inventory–II. Psychol Assess. APA Psyc Tests. 1996.

  53. https://www.mcgill.ca/neuro/.

  54. Tournoux TJ. Co-planar stereotaxic atlas of the human brain: 3-dimensional proportional system—an approach to cerebral imaging. New York: Thieme Medical Publishers; 1988.

    Google Scholar 

  55. Brodmann K. Vergleichende Lokalisationslehre der Grosshirnrinde in ihren Prinzipien dargestellt auf Grund des Zellenbaues: Barth; 1909.324 pages.

  56. Kingma D, Ba L. Adam: A Method for stochastic optimization. The 3rd International Conference for Learning Representations, San Diego, 2015.

  57. Kayalvizhi M. EEG signal extraction analysis techniques. Signal and Image Processing Techniques for the Development of Intelligent Healthcare Systems: Springer. 2021:223–39.

  58. Romanowicz K, Kozłowska K, Wichniak A. Psychomotor retardation in recurrent depression and the related factors. Adv Psychiat Neurol/Postępy Psychiatrii i Neurologii. 28(3):208–19.

Download references

Author information

Authors and Affiliations

  1. Department of Biomedical Engineering, Faculty of Electrical and Computer Engineering, University of Tabriz, Tabriz, Iran

    Shiva Asadzadeh, Tohid Yousefi Rezaii & Saeed Meshgini

  2. Department of Electrical and Computer Engineering, Ryerson University, Toronto, Canada

    Soosan Beheshti

Authors
  1. Shiva Asadzadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tohid Yousefi Rezaii
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Soosan Beheshti
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Saeed Meshgini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tohid Yousefi Rezaii.

Ethics declarations

Ethical Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Conflict of Interest

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Asadzadeh, S., Rezaii, T.Y., Beheshti, S. et al. Accurate Emotion Recognition Utilizing Extracted EEG Sources as Graph Neural Network Nodes. Cogn Comput 15, 176–189 (2023). https://doi.org/10.1007/s12559-022-10077-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-022-10077-5

Keywords

Search

Navigation