Skip to main content
Springer Nature Link
Log in

A Comprehensive Review on Sentiment Perception Using Electroencephalography (EEG)

  • Review Article
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Identification of sentiments using EEG is challenging and functional research area in human–computer interaction. In recent years, considerable work is done regarding detection and classification of emotions in affective computing. This review paper aims to comprehensively summarize various techniques and methods last updated in this field. The results of various techniques are mapped quantitatively by evaluating famous publications like sentiment analysis using EEG signals, detection of emotions using bio-potential signals, linear discriminant analysis (LDA) classifiers, identification of emotions from multichannel EEG through deep forest. It delineates an integrated informative approach in which various aspects of statistics from continuum of structured data sources are placed together. The most recent publications are inspected to examine the reliable approach for detection of sentiments. Moreover, there are some specific inputs for each research which are helpful to improve the performance of existing approaches in practical applications. The analysis and comparison of all the methods show that identification of emotions using multi-channel EEG through deep forest, facial expression recognition and recognition of sentiments using classifiers including LDA and SVM show best accuracies than state-of-art methods. We analyzed results of standard signals to measure the rare artifact-eliminated EEG signals. DEAP and DREAMER are challenging datasets which are being used in most of the techniques for detection and analysis of sentiments. Other datasets like GAPED, MANHOB HCl, ACSERTAINL, MULSEMEDIA and DECAF are also analyzed in various methods of sentiment analysis. The main target of this survey is to provide nearly full image of techniques regarding analysis of sentiments (detection of emotions and building resources). It is observed that the traditional procedure of feature extraction is followed along with addition of some new features in recent publications. Among all methods, it is observed that deep forest model for analysis of sentiments is oblivious to hyper-parameter settings that lead reducing complexity of recognition of sentiments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Sarvakar K et al. Facial emotion recognition using convolutional neural networks. Materials Today: Proceedings, 2021.

  2. Takahashi K. Remarks on emotion recognition from bio-potential signals. In: 2nd International conference on Autonomous Robots and Agents. 2004. Citeseer.

  3. Matlovic T et al. Emotions detection using facial expressions recognition and EEG. In: 2016 11th international workshop on semantic and social media adaptation and personalization (SMAP). 2016. IEEE.

  4. Poria S, et al. Fusing audio, visual and textual clues for sentiment analysis from multimodal content. Neurocomputing. 2016;174:50–9.

    Article  Google Scholar 

  5. Santamaria-Granados L, et al. Using deep convolutional neural network for emotion detection on a physiological signals dataset (AMIGOS). IEEE Access. 2018;7:57–67.

    Article  Google Scholar 

  6. Bhardwaj A et al. Classification of human emotions from EEG signals using SVM and LDA Classifiers. In: 2015 2nd International conference on signal processing and integrated networks (SPIN). 2015. IEEE.

  7. Thammasan N, et al. Continuous music-emotion recognition based on electroencephalogram. IEICE Trans Inf Syst. 2016;99(4):1234–41.

    Article  Google Scholar 

  8. Black MJ, Yacoob Y. Recognizing facial expressions in image sequences using local parameterized models of image motion. Int J Comput Vis. 1997;25(1):23–48.

    Article  Google Scholar 

  9. Zhang Y, Ji X, Zhang S. An approach to EEG-based emotion recognition using combined feature extraction method. Neurosci Lett. 2016;633:152–7.

    Article  Google Scholar 

  10. Langner O, et al. Presentation and validation of the Radboud Faces Database. Cogn Emot. 2010;24(8):1377–88.

    Article  Google Scholar 

  11. Niemic C. Studies of emotion: a theoretical and empirical review of psychophysiological studies of emotion. 2004.

  12. Yang W, et al. Effects of sound frequency on audiovisual integration: an event-related potential study. PLoS ONE. 2015;10(9): e0138296.

    Article  Google Scholar 

  13. Soleymani M, et al. Analysis of EEG signals and facial expressions for continuous emotion detection. IEEE Trans Affect Comput. 2015;7(1):17–28.

    Article  Google Scholar 

  14. Wang X-W, Nie D, Lu B-L. Emotional state classification from EEG data using machine learning approach. Neurocomputing. 2014;129:94–106.

    Article  Google Scholar 

  15. Mohammadi Z, Frounchi J, Amiri M. Wavelet-based emotion recognition system using EEG signal. Neural Comput Appl. 2017;28(8):1985–90.

    Article  Google Scholar 

  16. Yin Z, et al. Recognition of emotions using multimodal physiological signals and an ensemble deep learning model. Comput Methods Progr Biomed. 2017;140:93–110.

    Article  Google Scholar 

  17. Ramirez R, Vamvakousis Z. Detecting emotion from EEG signals using the emotive epoc device. In: International conference on brain informatics. Springer, Berlin; 2012.

  18. Bos DO. EEG-based emotion recognition. Influenc Visual Audit Stimuli. 2006;56(3):1–17.

    Google Scholar 

  19. Bynion T-M, Feldner MT. Self-assessment manikin. In: Encyclopedia of personality and individual differences. Berlin: Springer; 2017. p. 1–3.

    Google Scholar 

  20. Tseng Y-L, et al. Voluntary attention in Asperger’s syndrome: brain electrical oscillation and phase-synchronization during facial emotion recognition. Res Autism Spectr Disord. 2015;13:32–51.

    Article  Google Scholar 

  21. Hernández-Travieso JG et al. Expression detector system based on facial images. In: BIOSIGNALS. 2013.

  22. Raheel A, Majid M, Anwar SM. DEAR-MULSEMEDIA: dataset for emotion analysis and recognition in response to multiple sensorial media. Inf Fusion. 2021;65:37–49.

    Article  Google Scholar 

  23. Goswamil S, Poray J. Human computer interaction for sentiment analysis and opinion mining: a review. In: 2016 International conference on computer, electrical & communication engineering (ICCECE). 2016. IEEE.

  24. Yang D, et al. Decoding visual motions from EEG using attention-based RNN. Appl Sci. 2020;10(16):5662.

    Article  Google Scholar 

  25. Soroush MZ, et al. A review on EEG signals based emotion recognition. Int Clin Neurosci J. 2017;4(4):118.

    Article  MathSciNet  Google Scholar 

  26. Hidalgo-Muñoz AR, et al. Spectral turbulence measuring as feature extraction method from EEG on affective computing. Biomed Signal Process Control. 2013;8(6):945–50.

    Article  Google Scholar 

  27. Lin YP et al. EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. In: 2009 IEEE international conference on acoustics, speech and signal processing. 2009. IEEE.

  28. Latif A, et al. Content-based image retrieval and feature extraction: a comprehensive review. Math Probl Eng. 2019;2019:1.

    Article  Google Scholar 

  29. Shabbir A, et al. Detection of glaucoma using retinal fundus images: a comprehensive review. Math Biosci Eng. 2021;18(3):2033–76.

    Article  Google Scholar 

  30. Rasheed A, et al. Fabric defect detection using computer vision techniques: a comprehensive review. Math Probl Eng. 2020;2020:1.

    Article  Google Scholar 

  31. Aslam MA et al. Image classification based on mid-level feature fusion. In: 2019 15th International conference on emerging technologies (ICET). 2019. IEEE.

  32. Habimana O, et al. Sentiment analysis using deep learning approaches: an overview. Sci China Inf Sci. 2020;63(1):1–36.

    Article  Google Scholar 

  33. Ko BC. A brief review of facial emotion recognition based on visual information. Sensors. 2018;18(2):401.

    Article  Google Scholar 

  34. Ebrahimi Kahou S et al. Recurrent neural networks for emotion recognition in video. In: Proceedings of the 2015 ACM on international conference on multimodal interaction. 2015.

  35. Walecki R, et al. Deep structured learning for facial expression intensity estimation. Image Vis Comput. 2017;259:143–54.

    Google Scholar 

  36. Joseph A, Geetha P. Facial emotion detection using modified eyemap–mouthmap algorithm on an enhanced image and classification with tensorflow. Vis Comput. 2020;36(3):529–39.

    Article  Google Scholar 

  37. Kim DH et al. A facial expression imitation system for the primitive of intuitive human-robot interaction. In: Human robot interaction. 2007. IntechOpen.

  38. Huber E. Evolution of facial musculature and facial expression. 1931.

  39. Narayan Y. Direct comparison of SVM and LR classifier for SEMG signal classification using TFD features. Mater Today Proc. 2021;45:3543–6.

    Article  Google Scholar 

  40. Murphy KP. Naive bayes classifiers. Univ Br Columbia. 2006;18(60):1–8.

    Google Scholar 

  41. Valueva MV, et al. Application of the residue number system to reduce hardware costs of the convolutional neural network implementation. Math Comput Simul. 2020;177:232–43.

    Article  MathSciNet  MATH  Google Scholar 

  42. Gers FA, Schmidhuber J, Cummins F. Learning to forget: continual prediction with LSTM. Neural Comput. 2000;12(10):2451–71.

    Article  Google Scholar 

  43. Britz D. Recurrent neural network tutorial, part 4 implementing a gru/lstm rnn with python and theano. http://www.wildml.com/2015/10/recurrent-neural-network-tutorial-part-4-implementing-a-grulstm-rnn-with-python-and-theano, 2015.

  44. Tripathi S et al. Using deep and convolutional neural networks for accurate emotion classification on DEAP dataset. In: Proceedings of the thirty-first AAAI conference on artificial intelligence. 2017.

  45. Kim MK, et al. A review on the computational methods for emotional state estimation from the human EEG. Comput Math Methods Med. 2013;2013:1.

    MathSciNet  Google Scholar 

  46. Jenke R, Peer A, Buss M. Feature extraction and selection for emotion recognition from EEG. IEEE Trans Affect Comput. 2014;5(3):327–39.

    Article  Google Scholar 

  47. Frantzidis CA, et al. Toward emotion aware computing: an integrated approach using multichannel neurophysiological recordings and affective visual stimuli. IEEE Trans Inf Technol Biomed. 2010;14(3):589–97.

    Article  Google Scholar 

  48. Hausdorff JM, et al. Dynamic markers of altered gait rhythm in amyotrophic lateral sclerosis. J Appl Physiol. 2000;88:2045.

    Article  Google Scholar 

  49. Ansari-Asl K, Chanel G, Pun T. A channel selection method for EEG classification in emotion assessment based on synchronization likelihood. In: 2007 15th European signal processing conference. 2007. IEEE.

  50. Khosrowabadi R, bin Abdul Rahman AW. Classification of EEG correlates on emotion using features from Gaussian mixtures of EEG spectrogram. In: Proceeding of the 3rd international conference on information and communication technology for the moslem world (ICT4M) 2010. 2010. IEEE.

  51. Sourina O, Liu Y. A fractal-based algorithm of emotion recognition from EEG using arousal-valence model. In: International conference on bio-inspired systems and signal processing. 2011. SCITEPRESS.

  52. Liu Y, Sourina O. Real-time fractal-based valence level recognition from EEG. In: Transactions on computational science XVIII. Springer; 2013. p. 101–20.

    Chapter  Google Scholar 

  53. Petrantonakis PC, Hadjileontiadis LJ. Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis. IEEE Trans Affect Comput. 2010;1(2):81–97.

    Article  Google Scholar 

  54. Boonyakitanont P, et al. A review of feature extraction and performance evaluation in epileptic seizure detection using EEG. Biomed Signal Process Control. 2020;57: 101702.

    Article  Google Scholar 

  55. Ahirwal MK, Kose MR. Emotion recognition system based on EEG signal: a comparative study of different features and classifiers. In: 2018 second international conference on computing methodologies and communication (ICCMC). 2018. IEEE.

  56. Mert A, Akan A. Emotion recognition from EEG signals by using multivariate empirical mode decomposition. Pattern Anal Appl. 2018;21(1):81–9.

    Article  MathSciNet  Google Scholar 

  57. Alhagry S, Fahmy AA, El-Khoribi RA. Emotion recognition based on EEG using LSTM recurrent neural network. Emotion. 2017;8(10):355–8.

    Google Scholar 

  58. Ackermann P et al. EEG-based automatic emotion recognition: feature extraction, selection and classification methods. In: 2016 IEEE 18th international conference on e-health networking, applications and services (Healthcom). 2016. IEEE.

  59. Ranganathan H, Chakraborty S, Panchanathan S. Multimodal emotion recognition using deep learning architectures. In: 2016 IEEE winter conference on applications of computer vision (WACV). 2016. IEEE.

  60. Cheng J, et al. Emotion recognition from multi-channel eeg via deep forest. IEEE J Biomed Health Informat. 2020;25:453.

    Article  Google Scholar 

  61. Katsigiannis S, Ramzan N. DREAMER: a database for emotion recognition through EEG and ECG signals from wireless low-cost off-the-shelf devices. IEEE J Biomed Health Inform. 2017;22(1):98–107.

    Article  Google Scholar 

  62. Quinlan JR. Induction of decision trees. Mach Learn. 1986;1(1):81–106.

    Article  Google Scholar 

  63. Yang Y et al. Continuous convolutional neural network with 3d input for eeg-based emotion recognition. In: International conference on neural information processing. 2018. Springer.

  64. Tao W, et al. EEG-based emotion recognition via channel-wise attention and self attention. IEEE Trans Affect Comput. 2020;2020:1.

    Article  Google Scholar 

  65. Yang Y et al. Emotion recognition from multi-channel EEG through parallel convolutional recurrent neural network. In: 2018 International joint conference on neural networks (IJCNN). 2018. IEEE.

  66. Zhang D, et al. A convolutional recurrent attention model for subject-independent eeg signal analysis. IEEE Signal Process Lett. 2019;26(5):715–9.

    Article  Google Scholar 

  67. Song T, et al. EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput. 2018;11(3):532–41.

    Article  Google Scholar 

  68. Miikkulainen R, et al. Evolving deep neural networks. In: Artificial intelligence in the age of neural networks and brain computing. Elsevier; 2019. p. 293–312.

    Chapter  Google Scholar 

  69. Liu S et al. Improve the generalization of emotional classifiers across time by using training samples from different days. In: 2016 38th Annual international conference of the IEEE engineering in medicine and biology society (EMBC). 2016. IEEE.

  70. Hassan MM, et al. Human emotion recognition using deep belief network architecture. Inf Fusion. 2019;51:10–8.

    Article  Google Scholar 

  71. Yuvaraj R, et al. Detection of emotions in Parkinson’s disease using higher order spectral features from brain’s electrical activity. Biomed Signal Process Control. 2014;14:108–16.

    Article  Google Scholar 

  72. Ahmed MA, Basori AH. The influence of beta signal toward emotion classification for facial expression control through EEG sensors. Procedia Soc Behav Sci. 2013;97:730–6.

    Article  Google Scholar 

  73. Goodman RN, et al. Stress, emotion regulation and cognitive performance: the predictive contributions of trait and state relative frontal EEG alpha asymmetry. Int J Psychophysiol. 2013;87(2):115–23.

    Article  Google Scholar 

  74. Vickers NJ. Animal communication: when i’m calling you, will you answer too? Curr Biol. 2017;27(14):R713–5.

    Article  Google Scholar 

  75. Lee G, et al. Emotion recognition based on 3D fuzzy visual and EEG features in movie clips. Neurocomputing. 2014;144:560–8.

    Article  Google Scholar 

  76. Mekler A, Gorbunov I, Gavrilov V. Systemic processes in the brain: the EEG study on the emotions of different hierarchical levels and signs. Int J Psychophysiol. 2014;94(2):191–191.

    Article  Google Scholar 

  77. Solomon B, et al. Negative affectivity and EEG asymmetry interact to predict emotional interference on attention in early school-aged children. Brain Cogn. 2014;87:173–80.

    Article  Google Scholar 

  78. Bong SZ, et al. Implementation of wavelet packet transform and non linear analysis for emotion classification in stroke patient using brain signals. Biomed Signal Process Control. 2017;36:102–12.

    Article  Google Scholar 

  79. Wei Y, Wu Y, Tudor J. A real-time wearable emotion detection headband based on EEG measurement. Sens Actuators A. 2017;263:614–21.

    Article  Google Scholar 

  80. Hoseingholizade S, Golpaygani MRH, Monfared AS. Studying emotion through nonlinear processing of EEG. Procedia Soc Behav Sci. 2012;32:163–9.

    Article  Google Scholar 

  81. Liu W, et al. Reinforcement online learning for emotion prediction by using physiological signals. Pattern Recogn Lett. 2018;107:123–30.

    Article  Google Scholar 

Download references

Funding

No funding was required for relative review article.

Author information

Authors and Affiliations

  1. Department of Software Engineering, Mirpur University of Science & Technology, Mirpur, 10250, AJK, Pakistan

    Iqra Ashraf Kiyani & Adeela Razaq

Authors
  1. Iqra Ashraf Kiyani
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Adeela Razaq
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Iqra Ashraf Kiyani.

Ethics declarations

Conflict of interest

The authors declare that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ashraf Kiyani, I., Razaq, A. A Comprehensive Review on Sentiment Perception Using Electroencephalography (EEG). SN COMPUT. SCI. 3, 245 (2022). https://doi.org/10.1007/s42979-022-01155-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-022-01155-4

Keywords