Skip to main content
SpringerLink
Log in

EEG processing in emotion recognition: inspired from a musical staff

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Common electroencephalograph (EEG) features have problems such as poor intrinsic correlation between characteristic quantities, low signal reproducibility and large data storage capacity, which lead to poor emotion recognition. To solve this problem, this paper proposes an EEG music model based on a musical staff. Firstly, this paper constructs a multi-channel EEG sensor network to measure the EEG of an individual under different emotional states, and establishes an EEG-Emotion mapping library for the individual. Then, the EEG is transformed by adaptive segmentation of the time-domain EEG signal using a musical staff model. The time-frequency characteristics of EEG, such as amplitude, contour and signal frequency, are expressed quantitatively in a standardized musical space. The results show that, while retaining the time-frequency features of EEG, the model has an average similarity of 0.9769 before and after reconstruction, a compression rate of 57.18%, and an emotional state recognition rate that is 10.1% higher than traditional features. The brain wave music generated by the model, as a media, provides reference for people to understand the change of emotional state, and also provides a new technical idea for the subsequent use of EEG music for emotional induction.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Aldridge A, Barnes E, Bethel CL, Carruth DW, Kocturova M, Pleva M, Juhar J, Ieee (2019) Accessible Electroencephalograms (EEGs): A Comparative Review with OpenBCI's Ultracortex Mark IV Headset. 2019 29th International Conference Radioelektronika. https://doi.org/10.1109/radioelek.2019.8733482

  2. Allman JM, Hakeem A, Erwin JM, Nimchinsky E, Hof P (2001) The anterior cingulate cortex - the evolution of an interface between emotion and cognition. In: Unity of knowledge: the convergence of natural and human science, vol 935. Annals of the New York Academy of Sciences. New York Acad Sciences, New York, pp 107–117. https://doi.org/10.1111/j.1749-6632.2001.tb03476.x

    Chapter  Google Scholar 

  3. Al-Salman W, Li Y, Wen P (2019) K-complexes detection in EEG signals using fractal and frequency features coupled with an ensemble classification model. Neuroscience 422:119–133. https://doi.org/10.1016/j.neuroscience.2019.10.034

    Article  Google Scholar 

  4. Anaya B, Vallorani A, Perez-Edgar K (2021) Dyadic behavioral synchrony between behaviorally inhibited and non-inhibited peers is associated with concordance in EEG frontal alpha asymmetry and Delta-Beta coupling. Biol Psychol 159:108018–108018. https://doi.org/10.1016/j.biopsycho.2021.108018

    Article  Google Scholar 

  5. Bablani A, Edla DR, Kuppili V, Ramesh D (2020) A multi stage EEG data classification using k-means and feed forward neural network. Clin Epidemiol Glob Health 8(3):718–724. https://doi.org/10.1016/j.cegh.2020.01.008

    Article  Google Scholar 

  6. Bai JJ, Luo K, Peng J, Shi JL, Wu Y, Feng LX, Li JQ, Wang YX (2017) Music Emotions Recognition by Cognitive Classification Methodologies. 2017 Ieee 16th international conference on cognitive informatics & cognitive computing. Ieee, New York

  7. Balasubramanian G, Kanagasabai A, Mohan J, Seshadri NPG (2018) Music induced emotion using wavelet packet decomposition-an EEG study. Biomed Signal Process Control 42:115–128. https://doi.org/10.1016/j.bspc.2018.01.015

    Article  Google Scholar 

  8. Blanchette I (2006) The effect of emotion on interpretation and logic in a conditional reasoning task. Mem Cogn 34(5):1112–1125. https://doi.org/10.3758/bf03193257

    Article  Google Scholar 

  9. Cai HS, Han JS, Chen YF, Sha XC, Wang ZY, Hu B, Yang J, Feng L, Ding ZJ, Chen YQ, Gutknecht J (2018) A pervasive approach to EEG-based depression detection. Complexity 2018:1–13. https://doi.org/10.1155/2018/5238028

    Article  Google Scholar 

  10. Cegar DD, Barreda-Angeles M, Kukolj D, Le Callet P (2020) Modelling effects of S3D visual discomfort in human emotional state using data mining techniques. Multimed Tools Appl 79(27–28):19803–19829. https://doi.org/10.1007/s11042-020-08844-3

    Article  Google Scholar 

  11. Choi S, Jiang Z (2008) Comparison of envelope extraction algorithms for cardiac sound signal segmentation. Expert Syst Appl 34(2):1056–1069. https://doi.org/10.1016/j.eswa.2006.12.015

    Article  Google Scholar 

  12. Davidson RJ (2004) What does the prefrontal cortex "do" in affect: perspectives on frontal EEG asymmetry research. Biol Psychol 67(1–2):219–233. https://doi.org/10.1016/j.biopsycho.2004.03.008

    Article  Google Scholar 

  13. Davidson RJ, Jackson DC, Kalin NH (2000) Emotion, plasticity, context, and regulation: perspectives from affective neuroscience. Psychol Bull 126(6):890–909. https://doi.org/10.1037//0033-2909.126.6.890

    Article  Google Scholar 

  14. Delorme A, Makeig S (2004) EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods 134(1):9–21. https://doi.org/10.1016/j.jneumeth.2003.10.009

    Article  Google Scholar 

  15. Fdez J, Guttenberg N, Witkowski O, Pasquali A (2021) Cross-subject EEG-based emotion recognition through neural networks with stratified normalization. Front Neurosci 15:626277–626277. https://doi.org/10.3389/fnins.2021.626277

    Article  Google Scholar 

  16. Geethanjali B, Adalarasu K, Jagannath M, Scshadri NPG (2019) Music-induced brain functional connectivity using EEG sensors: a study on Indian music. IEEE Sensors J 19(4):1499–1507. https://doi.org/10.1109/jsen.2018.2873402

    Article  Google Scholar 

  17. Hamada M, Zaidan BB, Zaidan AA (2018) A systematic review for human EEG brain signals based emotion classification, feature extraction, brain condition, group comparison. J Med Syst 42(9):25. https://doi.org/10.1007/s10916-018-1020-8

    Article  Google Scholar 

  18. Herrington JD, Mohanty A, Koven NS, Fisher JE, Stewart JL, Banich MT, Webb AG, Miller GA, Heller W (2005) Emotion-modulated performance and activity in left dorsolateral prefrontal cortex. Emotion 5(2):200–207. https://doi.org/10.1037/1528-3542.5.2.200

    Article  Google Scholar 

  19. Jatupaiboon N, Pan-ngum S, Israsena P (2013) Real-time EEG-based happiness detection system. Sci World J 2013:1–12. https://doi.org/10.1155/2013/618649

    Article  Google Scholar 

  20. Jie X, Rui C, Li L (2014) Emotion recognition based on the sample entropy of EEG. Bio-Med Mater Eng 24(1):1185–1192. https://doi.org/10.3233/bme-130919

    Article  MathSciNet  Google Scholar 

  21. Jirayucharoensak S, Pan-Ngum S, Israsena P (2014) EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci World J 2014:1–10. https://doi.org/10.1155/2014/627892

    Article  Google Scholar 

  22. Katsigiannis S, Ramzan N (2018) DREAMER: a database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE J Biomed Health Inform 22(1):98–107. https://doi.org/10.1109/jbhi.2017.2688239

    Article  Google Scholar 

  23. Khezri M, Firoozabadi M, Sharafat AR (2015) Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals. Comput Methods Prog Biomed 122(2):149–164. https://doi.org/10.1016/j.cmpb.2015.07.006

    Article  Google Scholar 

  24. Khosrowabadi R, Quek C, Ang KK, Wahab A (2014) ERNN: a biologically inspired feedforward neural network to discriminate emotion from EEG signal. IEEE Trans Neural Netw Learn Syst 25(3):609–620. https://doi.org/10.1109/tnnls.2013.2280271

    Article  Google Scholar 

  25. Koelsch S (2014) Brain correlates of music-evoked emotions. Nat Rev Neurosci 15(3):170–180. https://doi.org/10.1038/nrn3666

    Article  Google Scholar 

  26. Koelstra S, Muhl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis using physiological signals. IEEE Trans Affect Comput 3(1):18–31. https://doi.org/10.1109/t-affc.2011.15

    Article  Google Scholar 

  27. Kong R, Li JW, Orban C, Sabuncu MR, Liu HS, Schaefer A, Sun NB, Zuo XN, Holmes AJ, Eickhoff SB, Yeo BTT (2019) Spatial topography of individual-specific cortical networks predicts human cognition, personality, and emotion. Cereb Cortex 29(6):2533–2551. https://doi.org/10.1093/cercor/bhy123

    Article  Google Scholar 

  28. Kumar HIK, Harish BS, Darshan IIK (2019) Sentiment analysis on IMDb movie reviews using hybrid feature extraction method. Int J Interact Multimed Artif Intell 5(5):109–114. https://doi.org/10.9781/ijimai.2018.12.005

    Article  Google Scholar 

  29. Light SN, Coan JA, Zahn-Waxler C, Frye C, Goldsmith HH, Davidson RJ (2009) Empathy is associated with dynamic change in prefrontal brain electrical activity during positive emotion in children. Child Dev 80(4):1210–1231. https://doi.org/10.1111/j.1467-8624.2009.01326.x

    Article  Google Scholar 

  30. Lin YP, Wang CH, Jung TP, Wu TL, Jeng SK, Duann JR, Chen JH (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806. https://doi.org/10.1109/tbme.2010.2048568

    Article  Google Scholar 

  31. Liu YJ, Yu MJ, Zhao GZ, Song JJ, Ge Y, Shi YC (2018) Real-time movie-induced discrete emotion recognition from EEG signals. IEEE Trans Affect Comput 9(4):550–562. https://doi.org/10.1109/taffc.2017.2660485

    Article  Google Scholar 

  32. Moharreri S, Dabanloo NJ, Maghooli K (2018) Modeling the 2D space of emotions based on the poincare plot of heart rate variability signal. Biocybern Biomed Eng 38(4):794–809. https://doi.org/10.1016/j.bbe.2018.07.001

    Article  Google Scholar 

  33. Momennezhad A (2018) EEG-based emotion recognition utilizing wavelet coefficients. Multimed Tools Appl 77(20):27089–27106. https://doi.org/10.1007/s11042-018-5906-8

    Article  Google Scholar 

  34. Naji M, Firoozabadi M, Azadfallah P (2015) Emotion classification during music listening from forehead biosignals. Signal Image Video Process 9(6):1365–1375. https://doi.org/10.1007/s11760-013-0591-6

    Article  Google Scholar 

  35. Nawaz R, Cheah KH, Nisar H, Yap VV (2020) Comparison of different feature extraction methods for EEG-based emotion recognition. Biocybern Biomed Eng 40(3):910–926. https://doi.org/10.1016/j.bbe.2020.04.005

    Article  Google Scholar 

  36. Nguyen B, Nguyen D, Ma WL, Tran D, Ieee (2017) Wavelet transform and adaptive arithmetic coding techniques for EEG lossy compression. In: 2017 International Joint Conference on Neural Networks. IEEE International Joint Conference on Neural Networks (IJCNN). Ieee, New York, pp 3153–3160

  37. Petrantonakis PC, Hadjileontiadis LJ (2010) Emotion recognition from EEG using higher order crossings. IEEE Trans Inf Technol Biomed 14(2):186–197. https://doi.org/10.1109/titb.2009.2034649

    Article  Google Scholar 

  38. Picard RW, Vyzas E, Healey J (2001) Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans Pattern Anal Mach Intell 23(10):1175–1191. https://doi.org/10.1109/34.954607

    Article  Google Scholar 

  39. Raheel A, Anwar SM, Majid M (2019) Emotion recognition in response to traditional and tactile enhanced multimedia using electroencephalography. Multimed Tools Appl 78(10):13971–13985. https://doi.org/10.1007/s11042-018-6907-3

    Article  Google Scholar 

  40. Ramirez R, Planas J, Escude N, Mercade J, Farriols C (2018) EEG-based analysis of the emotional effect of music therapy on palliative care Cancer patients. Front Psychol 9:7. https://doi.org/10.3389/fpsyg.2018.00254

    Article  Google Scholar 

  41. Schmidt LA, Trainor LJ (2001) Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cognit Emot 15(4):487–500. https://doi.org/10.1080/0269993004200187

    Article  Google Scholar 

  42. Seth AK, Friston KJ (2016) Active interoceptive inference and the emotional brain. Philos Trans R Soc B-Biol Sci 371(1708):10. https://doi.org/10.1098/rstb.2016.0007

    Article  Google Scholar 

  43. Soleymani M, Lichtenauer J, Pun T, Pantic M (2012) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3(1):42–55. https://doi.org/10.1109/t-affc.2011.25

    Article  Google Scholar 

  44. Srinivasan K, Dauwels J, Reddy MR (2013) Multichannel EEG compression: wavelet-based image and volumetric coding approach. IEEE J Biomed Health Inform 17(1):113–120. https://doi.org/10.1109/titb.2012.2194298

    Article  Google Scholar 

  45. Turabzadeh S, Meng HY, Swash RM, Pleva M, Juhar J (2018) Facial expression emotion detection for real-time embedded systems. Technologies 6(1):18. https://doi.org/10.3390/technologies6010017

    Article  Google Scholar 

  46. Xia M, Li T, Shu TX, Wan JF, de Silva CW, Wang ZR (2019) A two-stage approach for the remaining useful life prediction of bearings using deep neural networks. IEEE Trans Ind Inform 15(6):3703–3711. https://doi.org/10.1109/tii.2018.2868687

    Article  Google Scholar 

  47. Yuvaraj R, Murugappan M (2016) Hemispheric asymmetry non-linear analysis of EEG during emotional responses from idiopathic Parkinson's disease patients. Cogn Neurodyn 10(3):225–234. https://doi.org/10.1007/s11571-016-9375-3

    Article  Google Scholar 

Download references

Funding

Thanks for the project supported by the national natural science foundation of China (grant no. 61573073).

Author information

Authors and Affiliations

  1. Key Laboratory for Optoelectronic Technology and System of the Education Ministry of China, College of Optoelectronic Engineering, Chongqing University, No. 174 Shazhengjie, Shapingba, Chongqing, 400044, China

    Yinsheng Li & Wei Zheng

  2. College of Information Science and Engineering, Henan University of Technology, Zhengzhou, 450001, China

    Yinsheng Li

Authors
  1. Yinsheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

YL and WZ worked on the methods section, results, discussion, and conclusion. WZ contributed in the introduction section of the manuscript and literature section.

Corresponding author

Correspondence to Wei Zheng.

Ethics declarations

Conflict of interest

No conflict of interest.

Ethics approval

All methods in this study were approved by the Ethics Committee of Chongqing Tumor Hospital (Approval No.: 005, 2019), and all experimental procedures were in accordance with the ethical guidelines and Helsinki declaration stipulated by the ministry of health, labor and welfare (BMJ 1991; 302: 1194).

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Consent for publication

The participants have consented to the submission of the report to the journal.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, Y., Zheng, W. EEG processing in emotion recognition: inspired from a musical staff. Multimed Tools Appl 82, 4161–4180 (2023). https://doi.org/10.1007/s11042-022-13405-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13405-x

Keywords

Search

Navigation