Skip to main content

Advertisement

Springer Nature Link
Log in

An improved graph convolutional neural network for EEG emotion recognition

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Dynamic uncertainty of the relationship among brain regions is an important limiting factor in electroencephalography (EEG)-based emotion recognition. This uncertainty stems from individual differences and emotional volatility, which needs further in-depth study. In this paper, we propose a new emotion recognition method, which is named graph convolutional neural network with spatio-temporal modeling and long short-term memory (STLGCNN). The proposed method aims to address the instability of emotion intensity and underutilization of EEG biotopological information. The method consists of an attention module, a bi-directional long short-term memory network (BiLSTM), a graph convolutional neural network (GCNN) and a long short-term memory module (LSTM). The attention mechanism is utilized to reveal correlations between different time periods and to reduce emotional temporal volatility. The BiLSTM is employed to learn spatio-temporal features. Then, the GCNN learns the biotopological information of multi-channel EEG signals and extracts effective graph domain features. These features are then fed into the LSTM to integrate the graph-domain information and extract valid temporal information. To verify the effectiveness of the STLGCNN method, we conducted experiments on the DEAP and SEED datasets. The average accuracies on the two datasets are 93.95 and 96.78%, respectively. The results show that the STLGCNN method has better performance than existing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

The datasets analyzed in this study are available in the DEAP and SEED databases at: https://www.eecs.qmul.ac.uk/mmv/datasets/deap/ and https://bcmi.sjtu.edu.cn/ seed/index.html, respectively

References

  1. Manasa G, Nirde KD, Gajre SS, Manthalkar R (2024) EEG signal-based classification of mental tasks using a one-dimensional convrest model. Neural Comput Appl 36:9053–9072. https://doi.org/10.1007/s00521-024-09550-z

    Article  Google Scholar 

  2. Huang D, Chen S, Liu C, Zheng L, Jiang D (2021) Differences first in asymmetric brain: a bi-hemisphere discrepancy convolutional neural network for EEG emotion recognition. Neurocomputing 448:140–151. https://doi.org/10.1016/j.neucom.2021.03.105

    Article  Google Scholar 

  3. Shanmugam S, Dharmar S (2023) A CNN-LSTM hybrid network for automatic seizure detection in EEG signals. Neural Comput Appl 35:20605–20617. https://doi.org/10.1007/s00521-023-08832-2

    Article  Google Scholar 

  4. Xue Y, Zheng W, Zong Y, Chang H, Jiang X (2022) Adaptive hierarchical graph convolutional network for eeg emotion recognition. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp 1–8 . https://doi.org/10.1109/IJCNN55064.2022.9892411

  5. Yang Y, Wu Q, Qiu M, Wang Y, Chen X (2018) Emotion recognition from multi-channel eeg through parallel convolutional recurrent neural network. In: 2018 International Joint Conference on Neural Networks (IJCNN), pp 1–7 . https://doi.org/10.1109/IJCNN.2018.8489331

  6. Lou Y, Wu R, Li J, Wang L, Li X, Chen G (2023) A learning convolutional neural network approach for network robustness prediction. IEEE Trans Cybern 537:4531–4544. https://doi.org/10.1109/TCYB.2022.3207878

    Article  Google Scholar 

  7. Yuan Q, Dai Y, Li G (2023) Exploration of english speech translation recognition based on the LSTM RNN algorithm. Neural Comput Appl 35:24961–24970. https://doi.org/10.1007/s00521-023-08462-8

    Article  Google Scholar 

  8. Zhou H, Shao L, Zhang H (2024) Srrnet: a transformer structure with adaptive 2-d spatial attention mechanism for cell phone-captured shopping receipt recognition. IEEE Trans Consumer Electron 701:3289–3298. https://doi.org/10.1109/TCE.2022.3229438

    Article  Google Scholar 

  9. Wu Z, Li Q, Zhang H (2022) Chain-structure echo state network with stochastic optimization: methodology and application. IEEE Trans Neural Netw Learn Syst 335:1974–1985. https://doi.org/10.1109/TNNLS.2021.3098866

    Article  Google Scholar 

  10. Yan H, Zhang H, Shi J, Ma J, Xu X (2023) Inspiration transfer for intelligent design: a generative adversarial network with fashion attributes disentanglement. IEEE Trans Consumer Electron 694:1152–1163. https://doi.org/10.1109/TCE.2023.3255831

    Article  Google Scholar 

  11. Bi J, Wang F, Yan X, Ping J, Wen Y (2022) Multi-domain fusion deep graph convolution neural network for EEG emotion recognition. Neural Comput Appl 35:22241–22255. https://doi.org/10.1007/s00521-022-07643-1

    Article  Google Scholar 

  12. Ye M, Chen CLP, Zhang T (2022) Hierarchical dynamic graph convolutional network with interpretability for eeg-based emotion recognition. IEEE Transactions on Neural Networks and Learning Systems 1–12. https://doi.org/10.1109/TNNLS.2022.3225855

  13. Song T, Zheng W, Song P, Cui Z (2020) EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput 11(3):532–541. https://doi.org/10.1109/TAFFC.2018.2817622

    Article  Google Scholar 

  14. Zhong P, Wang D, Miao C (2022) EEG-based emotion recognition using regularized graph neural networks. IEEE Trans Affect Comput 13(3):1290–1301. https://doi.org/10.1109/TAFFC.2020.2994159

    Article  Google Scholar 

  15. Feng L, Cheng C, Zhao M, Deng H, Zhang Y (2022) EEG-based emotion recognition using spatial-temporal graph convolutional LSTM with attention mechanism. IEEE J Biomed Health Inf 26(11):5406–5417. https://doi.org/10.1109/JBHI.2022.3198688

    Article  Google Scholar 

  16. Kuang D, Michoski C (2023) Attention with kernels for EEG-based emotion classification. Neural Comput Appl 36:5251–5266. https://doi.org/10.1007/s00521-023-09344-9

    Article  Google Scholar 

  17. Vaziri J, Farid D, Nazemi Ardakani M, Hosseini Bamakan SM, Shahlaei M (2023) A time-varying stock portfolio selection model based on optimized PSO-BILSTM and multi-objective mathematical programming under budget constraints. Neural Comput Appl 35:18445–18470. https://doi.org/10.1007/s00521-023-08669-9

    Article  Google Scholar 

  18. Mohammed KK, Hassanien AE, Afify HM (2023) Refinement of ensemble strategy for acute lymphoblastic leukemia microscopic images using hybrid CNN-GRU-BILSTM and MVSM classifier. Neural Comput Appl 35:17415–17427. https://doi.org/10.1007/s00521-023-08607-9

    Article  Google Scholar 

  19. Yu Y, Si X, Hu C, Zhang J (2019) A review of recurrent neural networks: LSTM cells and network architectures. Neural Comput 31(7):1235–1270. https://doi.org/10.1162/neco_a_01199

    Article  MathSciNet  Google Scholar 

  20. Zhang Y, Zhang Y, Wang S (2022) An attention-based hybrid deep learning model for EEG emotion recognition. Signal, Image Video Process 17:2305–2313. https://doi.org/10.1007/s11760-022-02447-1

    Article  Google Scholar 

  21. Mahmud MS, Saha O, Fattah SA (2022) An efficient bidirectional lstm-based deep neural network for automatic emotion recognition using eeg signal. In: 2022 12th International Conference on Electrical and Computer Engineering (ICECE), pp 417–420 . https://doi.org/10.1109/ICECE57408.2022.10088864

  22. Xiao G, Ye M, Xu B, Chen Z, Ren Q (2021) 4d attention-based neural network for EEG emotion recognition. Cogn Neurodyn 16:805–818. https://doi.org/10.1007/s11571-021-09751-5

    Article  Google Scholar 

  23. Levie R, Monti F, Bresson X, Bronstein MM (2019) Cayleynets: graph convolutional neural networks with complex rational spectral filters. IEEE Trans Signal Process 67(1):97–109. https://doi.org/10.1109/TSP.2018.2879624

    Article  MathSciNet  Google Scholar 

  24. Li Y, Chen J, Li F, Fu B, Wu H, Ji Y, Zhou Y, Niu Y, Shi G, Zheng W (2023) GMSS: graph-based multi-task self-supervised learning for EEG emotion recognition. IEEE Trans Affect Comput 14(3):2512–2525. https://doi.org/10.1109/TAFFC.2022.3170428

    Article  Google Scholar 

  25. Du G, Su J, Zhang L, Su K, Wang X, Teng S, Liu PX (2022) A multi-dimensional graph convolution network for EEG emotion recognition. IEEE Trans Instrum Meas 71:1–11. https://doi.org/10.1109/TIM.2022.3204314

    Article  Google Scholar 

  26. Gao Y, Fu X, Ouyang T, Wang Y (2022) EEG-GCN: spatio-temporal and self-adaptive graph convolutional networks for single and multi-view EEG-based emotion recognition. IEEE Signal Process Lett 29:1574–1578. https://doi.org/10.1109/LSP.2022.3179946

    Article  Google Scholar 

  27. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. CoRR arXiv:1706.03762

  28. Jiang B, Ding C, Luo B, Tang J (2013) Graph-laplacian pca: Closed-form solution and robustness. In: 2013 IEEE Conference on Computer Vision and Pattern Recognition, pp 3492–3498 . https://doi.org/10.1109/CVPR.2013.448

  29. Wu F, Jing XY, Wei P, Lan C, Ji Y, Jiang GP, Huang Q (2022) Semi-supervised multi-view graph convolutional networks with application to webpage classification. Inf Sci 591:142–154. https://doi.org/10.1016/j.neucom.2020.01.006

    Article  Google Scholar 

  30. Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31. https://doi.org/10.1109/T-AFFC.2011.15

    Article  Google Scholar 

  31. Zheng W-L, Lu B-L (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7(3):162–175. https://doi.org/10.1109/TAMD.2015.2431497

    Article  Google Scholar 

  32. Topic A, Russo M (2021) Emotion recognition based on EEG feature maps through deep learning network. Eng Sci Technol, Int J. https://doi.org/10.1016/j.neucom.2020.01.006

    Article  Google Scholar 

  33. Zhong Q, Zhu Y, Dongli C, Luwei X, Zhang H (2020) Electroencephalogram access for emotion recognition based on a deep hybrid network. Front Human Neurosci 14:589001. https://doi.org/10.3389/fnhum.2020.589001

    Article  Google Scholar 

  34. Wang Z, Tong Y, Heng X (2019) Phase-locking value based graph convolutional neural networks for emotion recognition. IEEE Access 7:93711–93722. https://doi.org/10.1109/ACCESS.2019.2927768

    Article  Google Scholar 

  35. Gao Z, Wang X, Yang Y, Li Y, Ma K, Chen G (2021) A channel-fused dense convolutional network for EEG-based emotion recognition. IEEE Trans Cogn Dev Syst 13(4):945–954. https://doi.org/10.1109/TCDS.2020.2976112

    Article  Google Scholar 

  36. Topic A, Russo M, Stella M, Šaric M (2022) Emotion recognition using a reduced set of EEG channels based on holographic feature maps. Sensors 22:3248. https://doi.org/10.3390/s22093248

    Article  Google Scholar 

  37. Du X, Ma C, Zhang G, Li J, Lai Y-K, Zhao G, Deng X, Liu Y-J, Wang H (2022) An efficient LSTM network for emotion recognition from multichannel EEG signals. IEEE Trans Affect Comput 13(3):1528–1540. https://doi.org/10.1109/TAFFC.2020.3013711

    Article  Google Scholar 

  38. Tao W, Li C, Song R, Cheng J, Liu Y, Wan F, Chen X (2023) EEG-based emotion recognition via channel-wise attention and self attention. IEEE Trans Affect Comput 14(1):382–393. https://doi.org/10.1109/TAFFC.2020.3025777

    Article  Google Scholar 

  39. Wang Z, Liu Y, Zhang R, Zhang J, Guo X (2022) Eeg-based emotion recognition using partial directed coherence dense graph propagation. In: 2022 14th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA), pp 610–617 . https://doi.org/10.1109/ICMTMA54903.2022.00127

Download references

Acknowledgements

This work was supported in part by grants from the National Natural Science Foundation of China (Grant No. 62371341).

Author information

Authors and Affiliations

  1. Tianjin Key Laboratory of Wireless Mobile Communications and Power Transmission, Tianjin Normal University, Tianjin, 300387, China

    Bingyue Xu, Baiwei Sun & Yujie Wang

  2. College of Artificial Intelligence, Tianjin Normal University, Tianjin, 300387, China

    Xin Zhang

  3. College of Electronic and Communication Engineering, Tianjin Normal University, Tianjin, 300387, China

    Xiu Zhang

Authors
  1. Bingyue Xu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xin Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiu Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Baiwei Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yujie Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Bingyue Xu designed the method, implemented the experiments, and wrote the paper; Xin Zhang and Xiu Zhang reviewed the paper and worked in supervision; Baiwei Sun and Yujie Wang provided valuable feedback and did paper editing.

Corresponding author

Correspondence to Xin Zhang.

Ethics declarations

Conflict of interest

The authors have declared that there is no conflict of interest

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, B., Zhang, X., Zhang, X. et al. An improved graph convolutional neural network for EEG emotion recognition. Neural Comput & Applic 36, 23049–23060 (2024). https://doi.org/10.1007/s00521-024-10469-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-024-10469-8

Keywords