Abstract
In this paper, we proposed a real-time subject-dependent EEG-based emotion recognition algorithm and tested it on experiments’ databases and the benchmark database DEAP. The algorithm consists of two parts: feature extraction and data classification with Support Vector Machine (SVM). Use of a Fractal Dimension feature in combination with statistical and Higher Order Crossings (HOC) features gave results with the best accuracy and with adequate computational time. The features were calculated from EEG using a sliding window. The proposed algorithm can recognize up to 8 emotions such as happy, surprised, satisfied, protected, angry, frightened, unconcerned, and sad using 4 electrodes in real time. Two experiments with audio and visual stimuli were implemented, and the Emotiv EPOC device was used to collect EEG data.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Biosemi, http://www.biosemi.com
Emotiv, http://www.emotiv.com
American electroencephalographic society guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology 8(2), 200–202 (1991)
Accardo, A., Affinito, M., Carrozzi, M., Bouquet, F.: Use of the fractal dimension for the analysis of electroencephalographic time series. Biological Cybernetics 77(5), 339–350 (1997)
Aftanas, L.I., Lotova, N.V., Koshkarov, V.I., Popov, S.A.: Non-linear dynamical coupling between different brain areas during evoked emotions: An EEG investigation. Biological Psychology 48(2), 121–138 (1998)
Anderson, E.W., Potter, K.C., Matzen, L.E., Shepherd, J.F., Preston, G.A., Silva, C.T.: A user study of visualization effectiveness using EEG and cognitive load. Computer Graphics Forum 30(3), 791–800 (2011)
Arvaneh, M., Cuntai, G., Kai Keng, A., Chai, Q.: Optimizing the channel selection and classification accuracy in EEG-Based BCI. IEEE Transactions on Biomedical Engineering 58(6), 1865–1873 (2011)
Aspiras, T.H., Asari, V.K.: Log power representation of EEG spectral bands for the recognition of emotional states of mind. In: 8th International Conference on Information, Communications and Signal Processing (ICICS 2011), pp. 1–5 (2011)
Bechara, A., Damasio, H., Damasio, A.R.: Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex 10(3), 295–307 (2000)
Bolls, P.D., Lang, A., Potter, R.F.: The effects of message valence and listener arousal on attention, memory, and facial muscular responses to radio advertisements. Communication Research 28(5), 627–651 (2001)
Bos, D.O.: EEG-based emotion recognition (2006), http://hmi.ewi.utwente.nl/verslagen/capita-selecta/CS-Oude_Bos-Danny.pdf
Bradley, M.M.: Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49–59 (1994)
Bradley, M.M., Lang, P.J.: The international affective digitized sounds (2nd edn., IADS-2): Affective ratings of sounds and instruction manual. Tech. rep., University of Florida, Gainesville (2007)
Burgdorf, J., Panksepp, J.: The neurobiology of positive emotions. Neuroscience & Biobehavioral Reviews 30(2), 173–187 (2006)
Cao, M., Fang, G., Ren, F.: EEG-based emotion recognition in Chinese emotional words. In: Proceedings of CCIS 2011, pp. 452–456 (2011)
Chanel, G., Rebetez, C., Betrancourt, M., Pun, T.: Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Transactions on Systems, Man, and Cybernetics Part A: Systems and Humans 41(6), 1052–1063 (2011)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines (2001), http://www.csie.ntu.edu.tw/~cjlin/libsvm
Cristianini, N., Shawe-Taylor, J.: An introduction to Support Vector Machines: and other kernel-based learning methods. Cambridge University Press, New York (2000)
D’Alessandro, M., Esteller, R., Vachtsevanos, G., Hinson, A., Echauz, J., Litt, B.: Epileptic seizure prediction using hybrid feature selection over multiple intracranial EEG electrode contacts: a report of four patients. IEEE Transactions on Biomedical Engineering 50(5), 603–615 (2003)
Delorme, A., Makeig, S.: EEGLAB: An open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of Neuroscience Methods 134(1), 9–21 (2004)
Duvinage, M., Castermans, T., Dutoit, T., Petieau, M., Hoellinger, T., Saedeleer, C.D., Seetharaman, K., Cheron, G.: A P300-based quantitative comparison between the emotiv epoc headset and a medical EEG device. In: Proceedings of the 9th IASTED International Conference on Biomedical Engineering, pp. 37–42 (2012)
Gao, T., Wu, D., Huang, Y., Yao, D.: Detrended fluctuation analysis of the human EEG during listening to emotional music. J. Elect. Sci. Tech. Chin. 5, 272–277 (2007)
Hadjidimitriou, S., Zacharakis, A., Doulgeris, P., Panoulas, K., Hadjileontiadis, L., Panas, S.: Sensorimotor cortical response during motion reflecting audiovisual stimulation: evidence from fractal EEG analysis. Medical and Biological Engineering and Computing 48(6), 561–572 (2010)
Hadjidimitriou, S.K., Zacharakis, A.I., Doulgeris, P.C., Panoulas, K.J., Hadjileontiadis, L.J., Panas, S.M.: Revealing action representation processes in audio perception using fractal EEG analysis. IEEE Transactions on Biomedical Engineering 58(4), 1120–1129 (2011)
Higuchi, T.: Approach to an irregular time series on the basis of the fractal theory. Physica D: Nonlinear Phenomena 31(2), 277–283 (1988)
Hosseini, S.A., Khalilzadeh, M.A.: Emotional stress recognition system using EEG and psychophysiological signals: Using new labelling process of EEG signals in emotional stress state. In: 2010 International Conference on Biomedical Engineering and Computer Science (ICBECS), pp. 1–6. IEEE (2010)
Hou, X., Sourina, O.: Emotion-enabled haptic-based serious game for post stroke rehabilitation. In: Proceedings of VRST 2013, pp. 31–34 (2013)
Hsu, C.W., Chang, C.C., Lin, C.J.: A practical guide to support vector classification. Tech. rep., National Taiwan University, Taipei (2003)
Huang, D., Guan, C., Kai Keng, A., Haihong, Z., Yaozhang, P.: Asymmetric spatial pattern for EEG-based emotion detection. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–7 (2012)
Jones, N.A., Fox, N.A.: Electroencephalogram asymmetry during emotionally evocative films and its relation to positive and negative affectivity. Brain and Cognition 20(2), 280–299 (1992)
Kandel, E.R., Schwartz, J.H., Jessell, T.M., et al.: Principles of neural science, vol. 4. McGraw-Hill, New York (2000)
Kedem, B.: Time Series Analysis by Higher Order Crossing. IEEE Press, New York (1994)
Khosrowabadi, R., Wahab bin Abdul Rahman, A.: Classification of EEG correlates on emotion using features from gaussian mixtures of EEG spectrogram. In: 2010 International Conference on Information and Communication Technology for the Muslim World (ICT4M), pp. E102–E107. IEEE (2010)
Kil, D.H., Shin, F.B.: Pattern recognition and prediction with applications to signal characterization. AIP series in modern acoustics and signal processing. AIP Press, Woodbury (1996)
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP: A database for emotion analysis using physiological signals. IEEE Transactions on Affective Computing 3(1), 18–31 (2012)
Koelstra, S., Muhl, C., Soleymani, M., Lee, J.S., Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP dataset (2012), http://www.eecs.qmul.ac.uk/mmv/datasets/deap
Kringelbach, M.L.: The human orbitofrontal cortex: Linking reward to hedonic experience. Nature Reviews Neuroscience 6(9), 691–702 (2005)
Kulish, V., Sourin, A., Sourina, O.: Analysis and visualization of human electroencephalograms seen as fractal time series. Journal of Mechanics in Medicine and Biology 26(2), 175–188 (2006)
Kulish, V., Sourin, A., Sourina, O.: Human electroencephalograms seen as fractal time series: Mathematical analysis and visualization. Computers in Biology and Medicine 36(3), 291–302 (2006)
Lal, T.N., Schroder, M., Hinterberger, T., Weston, J., Bogdan, M., Birbaumer, N., Scholkopf, B.: Support vector channel selection in BCI. IEEE Transactions on Biomedical Engineering 51(6), 1003–1010 (2004)
Lang, P., Bradley, M., Cuthbert, B.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical report a-8, University of Florida, Gainesville, FL (2008)
Lin, Y.P., Wang, C.H., Jung, T.P., Wu, T.L., Jeng, S.K., Duann, J.R., Chen, J.H.: EEG-based emotion recognition in music listening. IEEE Transactions on Biomedical Engineering 57(7), 1798–1806 (2010)
Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based human emotion recognition and visualization. In: Proc. 2010 Int. Conf. on Cyberworlds, Singapore, pp. 262–269 (2010)
Liu, Y., Sourina, O., Nguyen, M.K.: Real-time EEG-based emotion recognition and its applications. In: Gavrilova, M.L., Tan, C.J.K., Sourin, A., Sourina, O. (eds.) Transactions on Computational Science XII. LNCS, vol. 6670, pp. 256–277. Springer, Heidelberg (2011)
Liu, Y., Sourina, O.: EEG-based emotion-adaptive advertising. In: Proc. ACII 2013, Geneva, pp. 843–848 (2013)
Liu, Y., Sourina, O.: EEG databases for emotion recognition. In: Proc. 2013 Int. Conf. on Cyberworlds, Japan (2013)
Liu, Y., Sourina, O.: Real-time fractal-based valence level recognition from EEG. In: Gavrilova, M.L., Tan, C.J.K., Kuijper, A. (eds.) Transactions on Computational Science XVIII. LNCS, vol. 7848, pp. 101–120. Springer, Heidelberg (2013)
Lutzenberger, W., Elbert, T., Birbaumer, N., Ray, W.J., Schupp, H.: The scalp distribution of the fractal dimension of the EEG and its variation with mental tasks. Brain Topography 5(1), 27–34 (1992)
Maragos, P., Sun, F.K.: Measuring the fractal dimension of signals: morphological covers and iterative optimization. IEEE Transactions on Signal Processing 41(1), 108–121 (1993)
Mauss, I.B., Robinson, M.D.: Measures of emotion: A review. Cognition and Emotion 23(2), 209–237 (2009)
Mehrabian, A.: Framework for a comprehensive description and measurement of emotional states. Genetic, Social, and General Psychology Monographs 121(3), 339–361 (1995)
Mehrabian, A.: Pleasure-arousal-dominance: A general framework for describing and measuring individual differences in temperament. Current Psychology 14(4), 261–292 (1996)
Noble, W.S.: What is a support vector machine? Nat. Biotech. 24(12), 1565–1567 (2006)
O’Regan, S., Faul, S., Marnane, W.: Automatic detection of EEG artefacts arising from head movements. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 6353–6356 (2010)
Petrantonakis, P.C., Hadjileontiadis, L.J.: Emotion recognition from EEG using higher order crossings. IEEE Transactions on Information Technology in Biomedicine 14(2), 186–197 (2010)
Petrantonakis, P.C., Hadjileontiadis, L.J.: Adaptive emotional information retrieval from EEG signals in the time-frequency domain. IEEE Transactions on Signal Processing 60(5), 2604–2616 (2012)
Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: Analysis of affective physiological state. IEEE Transactions on Pattern Analysis and Machine Intelligence 23(10), 1175–1191 (2001)
Pradhan, N., Narayana Dutt, D.: Use of running fractal dimension for the analysis of changing patterns in electroencephalograms. Computers in Biology and Medicine 23(5), 381–388 (1993)
Ranky, G.N., Adamovich, S.: Analysis of a commercial EEG device for the control of a robot arm. In: Proceedings of the 2010 IEEE 36th Annual Northeast Bioengineering Conference, pp. 1–2 (2010)
Russell, J.A.: Affective space is bipolar. Journal of Personality and Social Psychology 37(3), 345–356 (1979)
Schaaff, K., Schultz, T.: Towards emotion recognition from electroencephalographic signals. In: 3rd International Conference on Affective Computing and Intelligent Interaction and Workshops, ACII 2009, pp. 1–6 (2009)
Soleymani, M., Pantic, M., Pun, T.: Multimodal emotion recognition in response to videos. IEEE Transactions on Affective Computing 3(2), 211–223 (2012)
Sourina, O., Kulish, V.V., Sourin, A.: Novel tools for quantification of brain responses to music stimuli. In: Proc. of 13th International Conference on Biomedical Engineering, ICBME 2008, pp. 411–414 (2008)
Sourina, O., Liu, Y.: A fractal-based algorithm of emotion recognition from EEG using arousal-valence model. In: BIOSIGNALS, pp. 209–214 (2011)
Sourina, O., Liu, Y., Nguyen, M.K.: Real-time EEG-based emotion recognition for music therapy. Journal on Multimodal User Interfaces 5(1-2), 27–35 (2012)
Sourina, O., Sourin, A., Kulish, V.: EEG data driven animation and its application. In: Gagalowicz, A., Philips, W. (eds.) MIRAGE 2009. LNCS, vol. 5496, pp. 380–388. Springer, Heidelberg (2009)
Stam, C.J.: Nonlinear dynamical analysis of EEG and MEG: Review of an emerging field. Clinical Neurophysiology 116(10), 2266–2301 (2005)
Stytsenko, K., Jablonskis, E., Prahm, C.: Evaluation of consumer EEG device Emotiv EPOC. Poster session presented at MEi: CogSci Conference 2011, Ljubljana (2011)
Szily, E., Kéri, S.: Emotion-related brain regions. Ideggyógyászati Szemle 61(3-4), 77 (2008)
Takahashi, K.: Remarks on emotion recognition from multi-modal bio-potential signals. In: 2004 IEEE International Conference on Industrial Technology, vol. 3, pp. 1138–1143 (2004)
Vecchiato, G., Toppi, J., Astolfi, L., De Vico Fallani, F., Cincotti, F., Mattia, D., Bez, F., Babiloni, F.: Spectral EEG frontal asymmetries correlate with the experienced pleasantness of tv commercial advertisements. Medical and Biological Engineering and Computing 49(5), 579–583 (2011)
Wang, Q., Sourina, O., Nguyen, M.K.: EEG-based “serious” games design for medical applications. In: Proc. 2010 Int. Conf. on Cyberworlds, Singapore, pp. 270–276 (2010)
Wang, Q., Sourina, O., Nguyen, M.: Fractal dimension based neurofeedback in serious games. The Visual Computer 27(4), 299–309 (2011)
Zhang, Q., Lee, M.: Analysis of positive and negative emotions in natural scene using brain activity and gist. Neurocomputing 72(4-6), 1302–1306 (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Liu, Y., Sourina, O. (2014). Real-Time Subject-Dependent EEG-Based Emotion Recognition Algorithm. In: Gavrilova, M.L., Tan, C.J.K., Mao, X., Hong, L. (eds) Transactions on Computational Science XXIII. Lecture Notes in Computer Science, vol 8490. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-43790-2_11
Download citation
DOI: https://doi.org/10.1007/978-3-662-43790-2_11
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-43789-6
Online ISBN: 978-3-662-43790-2
eBook Packages: Computer ScienceComputer Science (R0)