Skip to main content

Advertisement

SpringerLink
Log in

A new approach for emotions recognition through EOG and EMG signals

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In this paper, an approach for emotion recognition using physiological signals has been development. The main purpose of this paper is to provide improved method for emotion recognition using horizontal electrooculogram, vertical electrooculogram, zygomaticus major electromyogram and trapezius electromyogram signals. Emotions are state of feeling that causes psychological and physical changes which affects our behaviour. Emotions are elicited using stimuli which include video, images and audio, etc. Here, emotions are elicited by audio-visual songs. For classification of emotions, time domain, frequency domain and entropy based features are extracted. These features are classified using support vector machine, naive Bayes and artificial neural network. The performance of each classifier and features is compared on the basis of accuracy, average precision and average recall. Primary contribution is the identification of time domain features as best features for EOG and EMG signals with ANN classifier to achieve maximum classification accuracy. Overall classification average accuracy (98%) of ANN is found best as compared to other classifiers. Global implications of this work is in utilization for artificial intelligence based models of human decision making systems by adding effect of emotions during decision making process modeling.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1
Fig.2
Fig.3
Fig.4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Yang, N., Muraleedharan, R., Kohl, J., Demirkol, I., Heinzelman, W., Sturge-Apple, M.: Speech-based emotion classification using multiclass SVM with hybrid kernel and thresholding fusion. In: 2012 IEEE Workshop on Spoken Language Technology, SLT 2012 - Proceedings. pp. 455–460 (2012)

  2. Pollreisz, D., Taherinejad, N.: A simple algorithm for emotion recognition, using physiological signals of a smart watch. In: Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS. pp. 2353–2356. Institute of Electrical and Electronics Engineers Inc. (2017)

  3. Salmam, F.Z., Madani, A., Kissi, M.: Fusing multi-stream deep neural networks for facial expression recognition. SIViP 13, 609–616 (2019). https://doi.org/10.1007/s11760-018-1388-4

    Article  Google Scholar 

  4. Yoo, H., Kim, M.Y., Kwon, O.: Emotional index measurement method for context-aware service. Expert Syst. Appl. 38, 785–793 (2011). https://doi.org/10.1016/j.eswa.2010.07.034

    Article  Google Scholar 

  5. Ahirwal, M.K., Kose, M.R.: Emotion Recognition System based on EEG signal: A Comparative Study of Different Features and Classifiers. In: Proceedings of the 2nd International Conference on Computing Methodologies and Communication, ICCMC 2018. pp. 472–476. Institute of Electrical and Electronics Engineers Inc. (2018)

  6. Basu, S., Bag, A., Aftabuddin, M., Mahadevappa, M., Mukherjee, J., Guha, R.: Effects of emotion on physiological signals. In: 2016 IEEE Annual India Conference, INDICON 2016. Institute of Electrical and Electronics Engineers Inc. (2017)

  7. Nakisa, B., Rastgoo, M.N., Tjondronegoro, D., Chandran, V.: Evolutionary computation algorithms for feature selection of EEG-based emotion recognition using mobile sensors. Expert Syst. Appl. 93, 143–155 (2018). https://doi.org/10.1016/j.eswa.2017.09.062

    Article  Google Scholar 

  8. Daimi, S.N., Saha, G.: Classification of emotions induced by music videos and correlation with participants’ rating. Expert Syst. Appl. 41, 6057–6065 (2014). https://doi.org/10.1016/j.eswa.2014.03.050

    Article  Google Scholar 

  9. Basar, M.D., Duru, A.D., Akan, A.: Emotional state detection based on common spatial patterns of EEG. SIViP 14, 473–481 (2020). https://doi.org/10.1007/s11760-019-01580-8

    Article  Google Scholar 

  10. Feng, H., Golshan, H.M., Mahoor, M.H.: A wavelet-based approach to emotion classification using EDA signals. Expert Syst. Appl. 112, 77–86 (2018). https://doi.org/10.1016/j.eswa.2018.06.014

    Article  Google Scholar 

  11. Zhang, Q., Chen, X., Zhan, Q., Yang, T., Xia, S.: Respiration-based emotion recognition with deep learning. Comput. Ind. 92–93, 84–90 (2017). https://doi.org/10.1016/j.compind.2017.04.005

    Article  Google Scholar 

  12. Xu, Y., Liu, G.Y.: A method of emotion recognition based on ECG signal. In: Proceedings of the 2009 International Conference on Computational Intelligence and Natural Computing, CINC 2009. pp. 202–205 (2009)

  13. Selvaraj, J., Murugappan, M., Wan, K., Yaacob, S.: Classification of emotional states from electrocardiogram signals: A non-linear approach based on hurst. BioMedical Engineering Online. 12, (2013). https://doi.org/10.1186/1475-925X-12-44

  14. Cheng, Y., Liu, G.Y., Zhang, H.L.: The research of EMG signal in emotion recognition based on TS and SBS algorithm. In: Proceedings - 3rd International Conference on Information Sciences and Interaction Sciences, ICIS 2010. pp. 363–366 (2010)

  15. Ahirwal, M.K., Kumar, A., Londhe, N.D., Bikrol, H.: Scalp connectivity networks for analysis of EEG signal during emotional stimulation. In: International Conference on Communication and Signal Processing, ICCSP 2016. pp. 592–596. Institute of Electrical and Electronics Engineers Inc. (2016)

  16. Perdiz, J., Pires, G., Nunes, U.J.: Emotional state detection based on EMG and EOG biosignals: A short survey. In: Bioengineering (ENBENG), 2017 IEEE 5th Portuguese Meeting on. pp. 1–4 (2017)

  17. Koelstra, S., Muhl, C., Soleymani, M., Jong-Seok Lee, Yazdani, A., Ebrahimi, T., Pun, T., Nijholt, A., Patras, I.: DEAP: A Database for Emotion Analysis ;Using Physiological Signals. IEEE Transactions on Affective Computing. 3, 18–31 (2012). https://doi.org/10.1109/T-AFFC.2011.15

  18. Greene, B.R., Faul, S., Marnane, W.P., Lightbody, G., Korotchikova, I., Boylan, G.B.: A comparison of quantitative EEG features for neonatal seizure detection. Clin. Neurophysiol. 119, 1248–1261 (2008). https://doi.org/10.1016/j.clinph.2008.02.001

    Article  Google Scholar 

  19. Jenke, R., Peer, A., Buss, M.: Feature extraction and selection for emotion recognition from EEG. IEEE Trans. Affect. Comput. 5, 327–339 (2014). https://doi.org/10.1109/TAFFC.2014.2339834

    Article  Google Scholar 

  20. Wang, L., Xue, W., Li, Y., Luo, M., Huang, J., Cui, W., Huang, C.: Automatic epileptic seizure detection in EEG signals using multi-domain feature extraction and nonlinear analysis. Entropy. 19, (2017). https://doi.org/10.3390/e19060222

  21. GaneshKumar, R.: Performance analysis of soft computing techniques for classifying cardiac arrhythmia. Indian J. Comput. Sci. Eng. 4, 459–465 (2014)

    Google Scholar 

  22. Jiang, P., Missoum, S., Chen, Z.: Optimal SVM parameter selection for non-separable and unbalanced datasets. Struct. Multidiscip. Optim. 50, 523–535 (2014). https://doi.org/10.1007/s00158-014-1105-z

    Article  Google Scholar 

  23. Wang, Z., Xue, X.: Multi-class support vector machine. In: Support Vector Machines Applications. pp. 23–48. Springer (2014)

  24. Ali, M.S.A.M., Shaari, N.F., Julai, N., Jahidin, A.H., Amiruddin, A.I., Noor, M.Z.H., Saaid, M.F.: Robust arrhythmia classifier using hybrid multilayered perceptron network. In: Proceedings - 2013 IEEE 9th International Colloquium on Signal Processing and its Applications, CSPA 2013. pp. 304–309 (2013)

  25. Ceylan, R., Özbay, Y.: Comparison of FCM, PCA and WT techniques for classification ECG arrhythmias using artificial neural network. Expert Syst. Appl. 33, 286–295 (2007). https://doi.org/10.1016/j.eswa.2006.05.014

    Article  Google Scholar 

  26. Ma, Y., Liang, S., Chen, X., Jia, C.: The approach to detect abnormal access behavior based on naive bayes algorithm. In: Proceedings - 2016 10th International Conference on Innovative Mobile and Internet Services in Ubiquitous Computing, IMIS 2016. pp. 313–315 (2016)

  27. Perez-Rosero, M.S., Rezaei, B., Akcakaya, M., Ostadabbas, S.: Decoding emotional experiences through physiological signal processing. In: 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). pp. 881–885. IEEE (2017)

  28. Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30, 2067–2083 (2008). https://doi.org/10.1109/TPAMI.2008.26

    Article  Google Scholar 

  29. Li, L., Chen, J.: Emotion Recognition Using Physiological Signals from Multiple Subjects. 2006 International Conference on Intelligent Information Hiding and Multimedia. 355–358 (2006). https://doi.org/10.1109/IIH-MSP.2006.265016

  30. Guendil, Z., Lachiri, Z., Maaoui, C., Pruski, A.: Emotion recognition from physiological signals using fusion of wavelet based features. In: Modelling, Identification and Control (ICMIC), 2015 7th International Conference on. pp. 1–6 (2015)

  31. Cruz, A., Garcia, D., Pires, G., Nunes, U.: Facial expression recognition based on EOG toward emotion detection for human-robot interaction. BIOSIGNALS 2015—8th International Conference on Bio-Inspired Systems and Signal Processing, Proceedings; Part of 8th International Joint Conference on Biomedical Engineering Systems and Technologies, BIOSTEC 2015. 31–37 (2015). https://doi.org/10.5220/0005187200310037

  32. Qiao, R., Qing, C., Zhang, T., Xing, X., Xu, X.: A novel deep-learning based framework for multi-subject emotion recognition. ICCSS 2017 - 2017 International Conference on Information, Cybernetics, and Computational Social Systems. 181–185 (2017). https://doi.org/10.1109/ICCSS.2017.8091408

  33. Zhuang, X., Rozgić, V., Crystal, M.: Compact unsupervised EEG response representation for emotion recognition. In: 2014 IEEE-EMBS International Conference on Biomedical and Health Informatics, BHI 2014. pp. 736–739 (2014)

  34. Torres-Valencia, C.A., Garcia-Arias, H.F., Lopez, M.A.A., Orozco-Gutiérrez, A.A.: Comparative analysis of physiological signals and electroencephalogram (EEG) for multimodal emotion recognition using generative models. In: 2014 XIX Symposium on Image, Signal Processing and Artificial Vision (STSIVA), pp. 1–5 (2014)

  35. Martınez, H.P.: Advancing Affect Modeling via Preference Learning and Unsupervised Feature Extraction. (Ph.D. Thesis) IT University of Copenhagen, Center for Computer Games Research, (2013)

  36. Xu, Y., Hubener, I., Seipp, A.K., Ohly, S., David, K.: From the lab to the real-world: An investigation on the influence of human movement on Emotion Recognition using physiological signals. 2017 IEEE International Conference on Pervasive Computing and Communications Workshops, PerCom Workshops 2017. 345–350 (2017). https://doi.org/10.1109/PERCOMW.2017.7917586

  37. Liu, W., Zheng, W.L., Lu, B.L.: Emotion recognition using multimodal deep learning. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics). 9948 LNCS, 521–529 (2016). https://doi.org/10.1007/978-3-319-46672-9_58

  38. Zangeneh Soroush, M., Maghooli, K., Setarehdan, S.K., Nasrabadi, A.M.: A novel EEG-based approach to classify emotions through phase space dynamics. SIViP 13, 1149–1156 (2019). https://doi.org/10.1007/s11760-019-01455-y

    Article  Google Scholar 

  39. Ahirwal, M.K., Kose, M.R.: Audio-visual stimulation based emotion classification by correlated EEG channels. Heal. Technol. 10, 7–23 (2020). https://doi.org/10.1007/s12553-019-00394-5

    Article  Google Scholar 

Download references

Acknowledgements

Authors are thankful to Sander Koelstra, Queen Mary University of London, United Kingdom and team for designing DEAP dataset, and providing it publically for academic research.

Author information

Authors and Affiliations

  1. National Institute of Technology Raipur, Raipur, India

    Mangesh Ramaji Kose

  2. Maulana Azad National Institute of Technology Bhopal, Bhopal, India

    Mitul Kumar Ahirwal

  3. Indian Institute of Information Technology Design and Manufacturing, Jabalpur, India

    Anil Kumar

Authors
  1. Mangesh Ramaji Kose
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mitul Kumar Ahirwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anil Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mitul Kumar Ahirwal.

Ethics declarations

Conflict of interest

We wish to confirm that there are no known conflicts of interest associated with this publication and there has been no significant financial support for this work that could have influenced its outcome.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kose, M.R., Ahirwal, M.K. & Kumar, A. A new approach for emotions recognition through EOG and EMG signals. SIViP 15, 1863–1871 (2021). https://doi.org/10.1007/s11760-021-01942-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-021-01942-1

Keywords

Search

Navigation