Abstract
For emotion recognition using EEG signals, the challenge is improving accuracy. This study proposes strategies that concentrate on incorporating emotion lateralization and ensemble learning approach to enhance the accuracy of EEG-based emotion recognition. In this paper, we obtained EEG signals from an EEG-based public emotion dataset with four classes (i.e. happy, sad, angry and relaxed). The EEG signal is acquired from pair asymmetry channels from left and right hemispheres. EEG features were extracted using a hybrid features extraction from three domains, namely time, frequency and wavelet. To demonstrate the lateralization, we performed a set of four experimental scenarios, i.e. without lateralization, right-/left-dominance lateralization, valence lateralization and others lateralization. For emotion classification, we use random forest (RF), which is known as the best classifier in ensemble learning. Tuning parameters in the RF model were done by grid search optimization. As a comparison of RF, we employed two prevalent algorithms in EEG, namely SVM and LDA. Emotion classification accuracy increased significantly from without lateralization to the valence lateralization using three pairs of asymmetry channel, i.e. T7–T8, C3–C4 and O1–O2. For the classification, the RF method provides the highest accuracy of 75.6% compared to SVM of 69.8% and LDA of 60.4%. In addition, the features of energy–entropy from wavelet are important for EEG emotion recognition. This study yields a significant performance improvement of EEG-based emotion recognition by the valence emotion lateralization. It indicates that happy and relaxed emotions are dominant in the left hemisphere, while angry and sad emotions are better recognized from the right hemisphere.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ahern GL, Schwartz GE (1985) Differential lateralization for positive and negative emotion in the human brain: EEG spectral analysis. Neuropsychologia 23:745–755
Altenmüller E, Schürmann K, Lim VK, Parlitz D (2002) Hits to the left, flops to the right: different emotions during listening to music are reflected in cortical lateralisation patterns. Neuropsychologia 40:2242–2256
Alves NT, Fukusima SS, Aznar-Casanova JA (2008) Models of brain asymmetry in emotional processing. Psychol Neurosci 1:63–66
Atkinson J, Campos D (2016) Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Expert Syst Appl 47:35–41. https://doi.org/10.1016/j.eswa.2015.10.049
Berk LS, Felten DL, Tan SA, Bittman BB, Westengard J (2001) Modulation of neuroimmune parameters during the eustress of humor-associated mirthful laughter. Altern Ther Health Med 7:62–76
Bernard S, Adam S, Heutte L (2012) Dynamic random forests. Pattern Recognit Lett 33:1580–1586
Bhatti AM, Majid M, Anwar SM, Khan B (2016) Human emotion recognition and analysis in response to audio music using brain signals. Comput Hum Behav 65:267–275. https://doi.org/10.1016/j.chb.2016.08.029
Borod JC, Kent J, Koff E, Martin C, Alpert M (1988) Facial asymmetry while posing positive and negative emotions: support for the right hemisphere hypothesis. Neuropsychologia 26:759–764. https://doi.org/10.1016/0028-3932(88)90013-9
Bos DO (2006) EEG-based emotion recognition. Influ Vis Audit Stimul 56:1–17
Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25:49–59
Breiman L (2001) Random forests. Mach Learn 45:5–32
Brosschot JF, Thayer JF (2003) Heart rate response is longer after negative emotions than after positive emotions. Int J Psychophysiol 50:181–187. https://doi.org/10.1016/S0167-8760(03)00146-6
Candra H, Yuwono M, Chai R, Handojoseno A, Elamvazuthi I, Nguyen HT, Su S (2015) Investigation of window size in classification of EEG-emotion signal with wavelet entropy and support vector machine. In: 2015 37th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 7250–7253
Castillo JC, Castro-González Á, Fernández-Caballero A, Latorre JM, Pastor JM, Fernández-Sotos A, Salichs MA (2016) Software architecture for smart emotion recognition and regulation of the ageing adult. Cogn Comput 8:357–367. https://doi.org/10.1007/s12559-016-9383-y
Davidson RJ, Schwartz GE, Saron C, Bennett J, Goleman DJ (1979) Frontal vs. parietal EEG asymmetry during positive and negative affect. Pyscophysiology 16:202–203
Demaree HA, Everhart DE, Youngstrom EA, Harrison DW (2005) Brain lateralization of emotional processing: historical roots and a future incorporating “Dominance”. Behav Cogn Neurosci Rev 4:3–20. https://doi.org/10.1177/1534582305276837
Diykh M, Li Y, Wen P (2016) EEG sleep stages classification based on time domain features and structural graph similarity. IEEE Trans Neural Syst Rehabil Eng 24:1159–1168. https://doi.org/10.1109/TNSRE.2016.2552539
Higuchi T (1988) Approach to an irregular time series on the basis of the fractal theory. Physica D 31:277–283
Hjorth B (1970) EEG analysis based on time domain properties. Electroencephalogr Clin Neurophysiol 29:306–310
Ibanez A, Melloni M, Huepe D, Helgiu E, Rivera-Rei A, Canales-Johnson A, Baker P, Moya A (2012) What event-related potentials (ERPs) bring to social neuroscience? Soc Neurosci 7:632–649. https://doi.org/10.1080/17470919.2012.691078
Jenke R, Peer A, Buss M (2014) Feature extraction and selection for emotion recognition from EEG. IEEE Trans Affect Comput 5:327–339. https://doi.org/10.1109/TAFFC.2014.2339834
Kaiser JF (1990) On a simple algorithm to calculate the ‘energy’ of a signal. In: 1990 international conference on acoustics, speech, and signal processing, 1990. ICASSP-90. IEEE, pp 381–384
Koelstra S, Muhl C, Soleymani M, Lee J-S, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3:18–31
Koh KB (1998) Emotion and immunity. J Psychosom Res 45:107–115
Kononenko I (2001) Machine learning for medical diagnosis: history, state of the art and perspective. Artif Intell Med 23:89–109
Li J, Zhang Z, He H (2017) Hierarchical convolutional neural networks for EEG-based emotion recognition. Cogn Comput 10:368–380. https://doi.org/10.1007/s12559-017-9533-x
Lin Y-P, Wang C-H, Jung T-P, Wu T-L, Jeng S-K, Duann J-R, Chen J-H (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57:1798–1806. https://doi.org/10.1109/TBME.2010.2048568
Liu Y, Sourina O, Nguyen MK (2011) Real-time EEG-based emotion recognition and its applications. In: Gavrilova ML, Tan CJK, Sourin A, Sourina O (eds) Transactions on computational science XII. Springer, Berlin, Heidelberg, pp 256–277
Mohammadi Z, Frounchi J, Amiri M (2016) Wavelet-based emotion recognition system using EEG signal. Neural Comput Appl. https://doi.org/10.1007/s00521-015-2149-8
Olofsson JK, Nordin S, Sequeira H, Polich J (2008) Affective picture processing: an integrative review of ERP findings. Biol Psychol 77:247–265. https://doi.org/10.1016/j.biopsycho.2007.11.006
Özerdem MS, Polat H (2017) Emotion recognition based on EEG features in movie clips with channel selection. Brain Informatics 4:241–252. https://doi.org/10.1007/s40708-017-0069-3
Pane ES, Hendrawan MA, Wibawa AD, Purnomo MH (2017) Identifying rules for electroencephalograph (EEG) emotion recognition and classification. In: 2017 5th international conference on instrumentation, communications, information technology, and biomedical engineering (ICICI-BME). IEEE, pp 167–172. https://doi.org/10.1109/ICICI-BME.2017.8537731
Petrantonakis PC, Hadjileontiadis LJ (2010) Emotion recognition from EEG using higher order crossings. IEEE Trans Inf Technol Biomed 14:186–197. https://doi.org/10.1109/TITB.2009.2034649
Pires P, Mendes L, Mendes J, Rodrigues R, Pereira A (2015) Integrated e-healthcare system for elderly support. Cogn Comput 8:368–384. https://doi.org/10.1007/s12559-015-9367-3
Pressman SD, Cohen S (2005) Does positive affect influence health? Psychol Bull 131:925–971. https://doi.org/10.1037/0033-2909.131.6.925
Purves D (ed) (2004) Neuroscience, 3rd edn. Sinauer Associates, Publishers, Sunderland
Qi Y (2012) Random forest for bioinformatics. In: Zhang C, Ma Y (eds) Ensemble machine learning. Springer, Boston, MA, pp 307–323
Schmidt LA, Trainor LJ (2001) Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cogn Emot 15:487–500. https://doi.org/10.1080/0269993004200187
Schwartz GE, Davidson RJ, Maer F (1975) Right hemisphere lateralization for emotion in the human brain: interactions with cognition. Science 190:286–288
Wager TD, Phan KL, Liberzon I, Taylor SF (2003) Valence, gender, and lateralization of functional brain anatomy in emotion: a meta-analysis of findings from neuroimaging. NeuroImage 19:513–531. https://doi.org/10.1016/S1053-8119(03)00078-8
Wang X-W, Nie D, Lu B-L (2014) Emotional state classification from EEG data using machine learning approach. Neurocomputing 129:94–106. https://doi.org/10.1016/j.neucom.2013.06.046
Wibawa AD, Purnomo MH, Marzuki A, Rumpa LD (2016) Physiological pattern of human state emotion based on ECG and pulse sensor. J Theor Appl Inf Technol 93(1)
Zhang J, Chen M, Zhao S, Hu S, Shi Z, Cao Y (2016a) ReliefF-based EEG sensor selection methods for emotion recognition. Sensors 16:1558. https://doi.org/10.3390/s16101558
Zhang Y, Ji X, Zhang S (2016b) An approach to EEG-based emotion recognition using combined feature extraction method. Neurosci Lett 633:152–157. https://doi.org/10.1016/j.neulet.2016.09.037
Zheng W (2017) Multichannel EEG-based emotion recognition via group sparse canonical correlation analysis. IEEE Trans Cogn Dev Syst 9:281–290. https://doi.org/10.1109/TCDS.2016.2587290
Zheng W-L, Lu B-L (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7:162–175. https://doi.org/10.1109/TAMD.2015.2431497
Acknowledgements
We would like to thank the Indonesia Endowment Fund for Education (LPDP), the Republic of Indonesia, which has provided funding through the Indonesia Education Scholarship granted to the first author.
Funding
This study was funded by Indonesia Endowment Fund for Education, contract number PRJ-2173/LPDP/2015.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants performed by any of the authors. The data being used in this article is the public dataset which can be accessed online and also cited as one of the references.
Informed consent
Informed consent was obtained from all individual participants included in the study performed by the dataset creator, not the author of these articles.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Handling editor: Luca Simione (CNR-ISTC, Rome).
Reviewers: Two researchers who prefer to remain anonymous.
Rights and permissions
About this article
Cite this article
Pane, E.S., Wibawa, A.D. & Purnomo, M.H. Improving the accuracy of EEG emotion recognition by combining valence lateralization and ensemble learning with tuning parameters. Cogn Process 20, 405–417 (2019). https://doi.org/10.1007/s10339-019-00924-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10339-019-00924-z