Skip to main content
SpringerLink
Log in

Subject independent emotion recognition system for people with facial deformity: an EEG based approach

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Emotion recognition from Electroencephalography (EEG) is a better choice for the people with facial deformity like where facial data is not accurate or not available for example burned or paralyzed faces. This research exploits the image processing capability of convolutional neural network (CNN) and proposes a CNN model to classify different emotions from the scalogram images of EEG data. Scalogram images from EEG obtained by applying continuous wavelet transform used for the study. The proposed model is subject independent where the objective is to extract emotion specific features from EEG data irrespective of the source of the data. The proposed emotion recognition model is evaluated on two benchmark public databases namely DEAP and SEED. In order to show the model as a purely subject independent one, the cross data base criteria is also used for evaluation. The various performance evaluation experiments show that the proposed model is comparable in terms of emotion classification accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Ackermann P, Kohlschein C, Bitsch JÁ, Wehrle K, Jeschke S (2016) EEG-based automatic emotion recognition: feature extraction, selection and classification methods. In: e-Health networking, applications and services (Healthcom), 2016 IEEE 18th international conference on. IEEE, pp 1–6

  • Alarcao SM, Fonseca MJ (2017) Emotions recognition using EEG signals: a survey. IEEE Trans Affect Comput 10(3):374–393

    Article  Google Scholar 

  • Bolós VJ, Benítez R (2014) The wavelet scalogram in the study of time series. In: Advances in differential equations and applications. Springer, Cham, pp 147–154

  • Bradley MM, Lang PJ (1994) Measuring emotion: the self-assessment manikin and the semantic differential. J Behav Ther Exp Psychiatry 25(1):49–59

    Article  Google Scholar 

  • Fazli S, Popescu F, Danóczy M, Blankertz B, Müller KR, Grozea C (2009) Subject-independent mental state classification in single trials. Neural Netw 22(9):1305–1312

    Article  Google Scholar 

  • Jayaram V, Alamgir M, Altun Y, Scholkopf B, Grosse-Wentrup M (2016) Transfer learning in brain-computer interfaces. IEEE Comput Intell Mag 11(1):20–31

    Article  Google Scholar 

  • Jirayucharoensak S, Pan-Ngum S, Israsena P (2014) EEG-based emotion recognition using deep learning network with principal component based covariate shift adaptation. Sci World J 2014. https://doi.org/10.1155/2014/627892

    Article  Google Scholar 

  • Kareem A, Kijewski T (2002) Time-frequency analysis of wind effects on structures. J Wind Eng Ind Aerodyn 90(12–15):1435–1452

    Article  Google Scholar 

  • Klem GH, Lüders HO, Jasper HH, Elger C (1999) The ten-twenty electrode system of the International Federation. Electroencephalogr Clin Neurophysiol 52(3):3–6

    Google Scholar 

  • Koelstra S, Muhl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T et al (2011) Deap: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31

    Article  Google Scholar 

  • Lakhan P, Banluesombatkul N, Changniam V, Dhithijaiyratn R, Leelaarporn P, Boonchieng E et al (2019) Consumer grade brain sensing for emotion recognition. IEEE Sens J 19(21):9896–9907

    Article  Google Scholar 

  • Lan Z, Sourina O, Wang L, Scherer R, Müller-Putz GR (2018) Domain adaptation techniques for EEG-based emotion recognition: a comparative study on two public datasets. IEEE Trans Cogn Dev Syst 11(1):85–94

    Article  Google Scholar 

  • Lang PJ (1995) The emotion probe: studies of motivation and attention. Am Psychol 50(5):372

    Article  Google Scholar 

  • Li X, Song D, Zhang P, Yu G, Hou Y, Hu B (2016) Emotion recognition from multi-channel EEG data through convolutional recurrent neural network. In: 2016 IEEE international conference on bioinformatics and biomedicine (BIBM). IEEE, pp 352–359

  • Li Y, Zheng W, Zong Y, Cui Z, Zhang T, Zhou X (2018) A bi-hemisphere domain adversarial neural network model for EEG emotion recognition. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2018.2885474

    Article  Google Scholar 

  • Li J, Qiu S, Shen Y, Liu C, He H (2019) Multisource transfer learning for cross-subject EEG emotion recognition. IEEE Trans Cybern 20(7):3281–3293

    Google Scholar 

  • Liang Z, Oba S, Ishii S (2019) An unsupervised EEG decoding system for human emotion recognition. Neural Netw 116:257–268

    Article  Google Scholar 

  • Mauss IB, Robinson MD (2009) Measures of emotion: a review. Cogn Emot 23(2):209–237

    Article  Google Scholar 

  • Mert A, Akan A (2018) Emotion recognition from EEG signals by using multivariate empirical mode decomposition. Pattern Anal Appl 21(1):81–89

    Article  MathSciNet  Google Scholar 

  • Moon SE, Jang S, Lee JS (2018) Convolutional neural network approach for EEG-based emotion recognition using brain connectivity and its spatial information. In: 2018 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE, pp 2556–2560

  • Murugappan M, Ramachandran N, Sazali Y (2010) Classification of human emotion from EEG using discrete wavelet transform. J Biomed Sci Eng 3(04):390

    Article  Google Scholar 

  • Pandey P, Seeja KR (2019a) Emotional state recognition with EEG signals using subject independent approach. In: Mishra D, Yang XS, Unal A (eds) Data science and big data analytics. Lecture Notes on Data Engineering and Communications Technologies, vol 16. Springer, Singapore, pp 117–124

  • Pandey P, Seeja KR (2019b) Subject-independent emotion detection from EEG signals using deep neural network. In: International conference on innovative computing and communications. Springer, Singapore, pp 41–46

  • Pandey P, Seeja KR (2019c) Subject independent emotion recognition from EEG using VMD and deep learning. J King Saud Univ-Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2019.11.003

    Article  Google Scholar 

  • Petrantonakis PC, Hadjileontiadis LJ (2009) Emotion recognition from EEG using higher order crossings. IEEE Trans Inf Technol Biomed 14(2):186–197

    Article  Google Scholar 

  • Rayatdoost S, Soleymani M (2018). Cross-corpus eeg-based emotion recognition. In: 2018 IEEE 28th international workshop on machine learning for signal processing (MLSP). IEEE, pp 1–6

  • Russell JA (1980) A circumplex model of affect. J Pers Soc Psychol 39(6):1161–1178

    Article  Google Scholar 

  • Salama ES, El-Khoribi RA, Shoman ME, Shalaby MAW (2018) EEG-based emotion recognition using 3D convolutional neural networks. Int J Adv Comput Sci Appl 9(8):329–337

    Google Scholar 

  • Song T, Zheng W, Song P, Cui Z (2018) EEG emotion recognition using dynamical graph convolutional neural networks. IEEE Trans Affect Comput. https://doi.org/10.1109/TAFFC.2018.2817622

    Article  Google Scholar 

  • Sourina O, Liu Y (2011) A fractal-based algorithm of emotion recognition from EEG using arousal-valence model. In: International Conference on Bio-inspired Systems and Signal Processing, Vol 2. SCITEPRESS, pp 209–214

  • Wang XW, Nie D, Lu BL (2011) EEG-based emotion recognition using frequency domain features and support vector machines. In: International conference on neural information processing. Springer, Berlin, pp 734–743

  • Zheng WL, Lu BL (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7(3):162–175

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indira Gandhi Delhi Technical University for Women, Delhi, India

    Pallavi Pandey & K. R. Seeja

Authors
  1. Pallavi Pandey
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. K. R. Seeja
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to K. R. Seeja.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Pandey, P., Seeja, K.R. Subject independent emotion recognition system for people with facial deformity: an EEG based approach. J Ambient Intell Human Comput 12, 2311–2320 (2021). https://doi.org/10.1007/s12652-020-02338-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-02338-8

Keywords

Search

Navigation