Skip to main content
SpringerLink
Log in

Emotion Recognition from Electroencephalogram (EEG) Signals Using a Multiple Column Convolutional Neural Network Model

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Emotions are vital in human cognition and are essential for human survival. Emotion is often associated with smart decisions, interpersonal behavior, and, to some extent, intellectual cognition. From the recent literature on emotion recognition, we understand that the researchers are showing interest in creating meaningful "emotional" associations between humans and machines; there is a demand for accurate and scalable systems to detect human emotional states, as emotion recognition is needed to understand the mental status of such persons who cannot communicate their emotions, such as disabled people, mentally challenged persons, etc. Therefore, EEG signals provide a non-invasive method to identify the emotions of these disabled humans. The research community has recently been very interested in employing electroencephalography (EEG) for emotion classification since end-users have wearable EEG systems that may offer a portable, cheap, and straightforward technique for identifying emotions. Deep learning models have recently been extensively used to extract characteristics and recognize emotions from EEG recordings. Apart from that, various papers were reviewed in this research. This paper presented a multiple-column CNN network with a leaky ReLU activation function on the EEG brain wave and DEAP datasets. Multiple scalp electrode locations are used to collect EEG signals, and each electrode offers spatially unique data. The network can detect spatial correlations and extract characteristics that depict the spatial distribution of brain activity by employing a multiple-column CNN, which simultaneously processes signals from many channels. This makes it possible for the model to recognize emotions using the spatial information in EEG data. The result analysis was evaluated on different CNN models, and it was observed that an accuracy of 98.10% was achieved on the EEG brainwave dataset and 81% on the DEAP dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Data availability

Data will be made available on reasonable request.

References

  1. Kumari N, Anwar S, Bhattacharjee V. Time series-dependent feature of EEG signals for improved visually evoked emotion classification using EmotionCapsNet. Neural Comput Appl. 2022;34:13291–303. https://doi.org/10.1007/S00521-022-06942-X/METRICS.

    Article  Google Scholar 

  2. Chen T, Ju S, Ren F, Fan M, Gu Y. EEG emotion recognition model based on the LIBSVM classifier. Measurement. 2020;164: 108047.

    Article  Google Scholar 

  3. Gupta V, Chopda MD, Pachori RB. Cross-subject emotion recognition using flexible analytic wavelet transform from EEG signals. IEEE Sens J. 2019;19:2266–74. https://doi.org/10.1109/JSEN.2018.2883497.

    Article  Google Scholar 

  4. Qing C, Qiao R, Xu X, Cheng Y. Interpretable emotion recognition using EEG signals. IEEE Access. 2019;7:94160–70. https://doi.org/10.1109/ACCESS.2019.2928691.

    Article  Google Scholar 

  5. Pepa L, Spalazzi L, Capecci M, Ceravolo MG. Automatic emotion recognition in clinical scenario: a systematic review of methods. IEEE Trans Affect Comput. 2023;14:1675–95. https://doi.org/10.1109/TAFFC.2021.3128787.

    Article  Google Scholar 

  6. Zhou Y, Huang S, Xu Z, Wang P, Wu X, Zhang D. Cognitive workload recognition using EEG signals and machine learning: a review. IEEE Trans Cogn Dev Syst. 2022;14:799–818. https://doi.org/10.1109/TCDS.2021.3090217.

    Article  Google Scholar 

  7. Hwang S, Ki M, Hong K, Byun H. Subject-independent EEG-based emotion recognition using adversarial learning. Int Winter Conf Brain-Comput Interface BCI. 2020. https://doi.org/10.1109/BCI48061.2020.9061624.

    Article  Google Scholar 

  8. Tong L, Zhao J, Fu W. Emotion recognition and channel selection based on EEG signal. Int Conf Intell Comput Technol Autom ICICTA. 2018. https://doi.org/10.1109/ICICTA.2018.00031.

    Article  Google Scholar 

  9. Ruiz-Garcia A, Elshaw M, Altahhan A, Palade V. A hybrid deep learning neural approach for emotion recognition from facial expressions for socially assistive robots. Neural Comput Appl. 2018;29:359–73. https://doi.org/10.1007/S00521-018-3358-8.

    Article  Google Scholar 

  10. Mehendale N. Facial emotion recognition using convolutional neural networks (FERC). SN Appl Sci. 2020;2:1–8. https://doi.org/10.1007/S42452-020-2234-1/TABLES/3.

    Article  Google Scholar 

  11. Filippi P. Emotional voice intonation: a communication code at the origins of speech processing and word-meaning associations? J Nonverbal Behav. 2020;44:395–417. https://doi.org/10.1007/S10919-020-00337-Z.

    Article  Google Scholar 

  12. Izquierdo-Reyes J, Ramirez-Mendoza RA, Bustamante-Bello MR, Pons-Rovira JL, Gonzalez-Vargas JE. Emotion recognition for semi-autonomous vehicles framework. Int J Interact Des Manuf. 2018;12:1447–54. https://doi.org/10.1007/S12008-018-0473-9.

    Article  Google Scholar 

  13. Liao CY, Chen RC, Tai SK. Emotion stress detection using EEG signal and deep learning technologies. IEEE Int Conf Appl Syst Innov. 2018. https://doi.org/10.1109/ICASI.2018.8394414.

    Article  Google Scholar 

  14. Sugumar D, Vanathi PT. EEG Signal separation using improved EEMD-Fast IVA Algorithm. Asian J Res Soc Sci Hum. 2017;7:1230. https://doi.org/10.5958/2249-7315.2017.00239.8.

    Article  Google Scholar 

  15. Liu S, Zhao L, Wang X, Xin Q, Zhao J, Guttery DS, et al. Deep spatio-temporal representation and ensemble classification for attention deficit/hyperactivity disorder. IEEE Trans Neural Syst Rehabil Eng. 2021;29:1–10. https://doi.org/10.1109/TNSRE.2020.3019063.

    Article  Google Scholar 

  16. Maheshwari D, Ghosh SK, Tripathy RK, Sharma M, Acharya UR. Automated accurate emotion recognition system using rhythm-specific deep convolutional neural network technique with multi-channel EEG signals. Comput Biol Med. 2021;134: 104428. https://doi.org/10.1016/J.COMPBIOMED.2021.104428.

    Article  Google Scholar 

  17. Yang YX, Gao ZK, Wang XM, Li YL, Han JW, Marwan N, et al. A recurrence quantification analysis-based channel-frequency convolutional neural network for emotion recognition from EEG. Chaos. 2018. https://doi.org/10.1063/1.5023857.

    Article  Google Scholar 

  18. Arjun RAS, Panicker MR. Subject independent emotion recognition using EEG signals employing attention driven neural networks. Biomed Signal Process Control. 2022;75:103547. https://doi.org/10.1016/J.BSPC.2022.103547.

    Article  Google Scholar 

  19. Chao H, Dong L. emotion recognition using three-dimensional feature and convolutional neural network from multichannel EEG signals. IEEE Sens J. 2021;21:2024–34. https://doi.org/10.1109/JSEN.2020.3020828.

    Article  Google Scholar 

  20. Chen Y, Chang R, Guo J. Effects of data augmentation method borderline-SMOTE on emotion recognition of EEG signals based on convolutional neural network. IEEE Access. 2021;9:47491–502. https://doi.org/10.1109/ACCESS.2021.3068316.

    Article  Google Scholar 

  21. Wankhade SB, Doye DD. Hybrid hunt-based deep convolutional neural network for emotion recognition using EEG signals. Comput Methods Biomech Biomed Eng. 2022;25:1311–31. https://doi.org/10.1080/10255842.2021.2007889.

    Article  Google Scholar 

  22. Bagherzadeh S, Maghooli K, Shalbaf A, Maghsoudi A. Emotion recognition using effective connectivity and pre-trained convolutional neural networks in EEG signals. Cogn Neurodyn. 2022;16:1087–106. https://doi.org/10.1007/S11571-021-09756-0/TABLES/9.

    Article  Google Scholar 

  23. Pandey P, Seeja KR, et al. A One-Dimensional CNN Model for Subject Independent Emotion Recognition Using EEG Signals. In: Ashish K, et al., editors. International Conference on Innovative Computing and Communications: Proceedings of ICICC 2021, Volume 2. Singapore: Springer Singapore; 2022. p. 509–15.

    Chapter  Google Scholar 

  24. Bird JJ, Manso LJ, Ribiero EP, Ekart A, Faria DR (2018) “A study on mental state classification using eeg-based brain-machine interface,” in 9th International Conference on Intelligent Systems, IEEE

  25. Bird JJ, Faria DR, Manso LJ, Ekárt A, Buckingham CD. A deep evolutionary approach to bioinspired classifier optimisation for brain-machine interaction. Complexity. 2019;2019:1–4. https://doi.org/10.1155/2019/4316548.

    Article  Google Scholar 

  26. Awais M, Raza M, Singh N, Bashir K, Manzoor U, Islam SU, et al. LSTM-based emotion detection using physiological signals: iot framework for healthcare and distance learning in COVID-19. IEEE Internet Things J. 2021;8:16863–71. https://doi.org/10.1109/JIOT.2020.3044031.

    Article  Google Scholar 

  27. Patel P, Raghunandan R, Annavarapu RN. EEG-based human emotion recognition using entropy as a feature extraction measure. Brain Inform. 2021;8:1–13. https://doi.org/10.1186/S40708-021-00141-5/FIGURES/4.

    Article  Google Scholar 

  28. Sun M, Cui W, Yu S, Han H, Hu B, Li Y. A Dual-branch dynamic graph convolution based adaptive transformer feature fusion network for EEG emotion recognition. IEEE Trans Affect Comput. 2022;13:2218–28. https://doi.org/10.1109/TAFFC.2022.3199075.

    Article  Google Scholar 

  29. Gu X, Cai W, Gao M, Jiang Y, Ning X, Qian P. Multi-source domain transfer discriminative dictionary learning modeling for electroencephalogram-based emotion recognition. IEEE Trans Comput Soc Syst. 2022;9:1604–12. https://doi.org/10.1109/TCSS.2022.3153660.

    Article  Google Scholar 

  30. Li Q, Zhang T, Chen CLP, Yi K, Chen L. Residual GCB-net: residual graph convolutional broad network on emotion recognition. IEEE Trans Cogn Dev Syst. 2022. https://doi.org/10.1109/TCDS.2022.3147839.

    Article  Google Scholar 

  31. Song T, Zheng W, Song P, Cui Z. EEG Emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput. 2020;11:532–41. https://doi.org/10.1109/TAFFC.2018.2817622.

    Article  Google Scholar 

  32. Iyer A, Das SS, Teotia R, Maheshwari S, Sharma RR. CNN and LSTM based ensemble learning for human emotion recognition using EEG recordings. Multimed Tools Appl. 2023;82(4):4883–96.

    Article  Google Scholar 

  33. Atul Chauhan SKJ. Sharing image through visual secret sharing scheme using speech recognition method. IJAST. 2019;28(16):303–7.

    Google Scholar 

Download references

Funding

This study is not non-funding.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, National Institute of Technology Patna, Patna, 800005, India

    Sonu Kumar Jha, Somaraju Suvvari & Mukesh Kumar

Authors
  1. Sonu Kumar Jha
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Somaraju Suvvari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mukesh Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sonu Kumar Jha.

Ethics declarations

Conflict of interest

There is no conflict of interest among authors.

Research involving human and animal partcipants

This article does not contain any studies with human participants or animals performed by any authors.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Advanced Computing and Data Sciences” guest edited by Mayank Singh, Vipin Tyagi and P.K. Gupta.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jha, S.K., Suvvari, S. & Kumar, M. Emotion Recognition from Electroencephalogram (EEG) Signals Using a Multiple Column Convolutional Neural Network Model. SN COMPUT. SCI. 5, 213 (2024). https://doi.org/10.1007/s42979-023-02543-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-023-02543-0

Keywords

Search

Navigation