skip to main content
10.1145/3632410.3632417acmotherconferencesArticle/Chapter ViewAbstractPublication PagescomadConference Proceedingsconference-collections
short-paper

Joint Class Learning with Self Similarity Projection for EEG Emotion Recognition

Published: 04 January 2024 Publication History

Abstract

Brainwaves acquired through Electroencephalogram (EEG) capture emotions more reliably compared to other modalities for better human-computer interactions. Existing studies mainly focus on modelling tempo-spectral-spatial properties of EEG signals through transformation of the conventional features by using different deep learning techniques. However, learning intra and inter emotion class characteristics, have remained unexplored for EEG emotion recognition task. In this work, we propose a two-step learning approach to enable this. The model first performs a joint emotion class learning (jecl) to learn the class specific discriminative representation from EEG signals by retaining the similarities shared between different emotion classes. This is followed by self-similarity learning (ssl) which projects these embeddings into a joint discriminative latent subspace ensuring maximal covariance with the actual EEG input; we name our proposed approach as jec-ssl. The jec-ssl embeddings are further used to train a classifier for EEG emotion recognition task. Experiments with DEAP EEG emotion database show new state-of-the-art results in terms of accuracy and F1-score.

References

[1]
Soraia M Alarcao and Manuel J Fonseca. 2017. Emotions recognition using EEG signals: A survey. IEEE Transactions on Affective Computing 10, 3 (2017), 374–393.
[2]
John Atkinson and Daniel Campos. 2016. Improving BCI-based emotion recognition by combining EEG feature selection and kernel classifiers. Expert Systems with Applications 47 (2016), 35–41.
[3]
Forrest Sheng Bao, Xin Liu, Christina Zhang, 2011. PyEEG: an open source python module for EEG/MEG feature extraction. Computational intelligence and neuroscience 2011 (2011).
[4]
Gustavo EAPA Batista, Ana LC Bazzan, Maria Carolina Monard, 2003. Balancing training data for automated annotation of keywords: a case study.Wob 3 (2003), 10–8.
[5]
Andreas Baum and Laurent Vermue. 2019. Multiblock PLS: Block dependent prediction modeling for Python. Journal of Open Source Software 4, 34 (2019), 1190.
[6]
Andreas Baum and Laurent Vermue. Accessed June 2023. MB-PLS Python. https://mbpls.readthedocs.io/en/latest/mbpls.html.
[7]
Carlos Carreiras, Ana Priscila Alves, André Lourenço, Filipe Canento, Hugo Silva, Ana Fred, 2015–. BioSPPy: Biosignal Processing in Python. https://github.com/PIA-Group/BioSPPy/ [Online; accessed ].
[8]
Guillaume Chanel, Julien Kronegg, Didier Grandjean, and Thierry Pun. 2006. Emotion assessment: Arousal evaluation using EEG’s and peripheral physiological signals. In International workshop on multimedia content representation, classification and security. Springer, 530–537.
[9]
Hao Chao, Liang Dong, Yongli Liu, and Baoyun Lu. 2019. Emotion recognition from multiband EEG signals using CapsNet. Sensors 19, 9 (2019), 2212.
[10]
Nitesh V Chawla, Kevin W Bowyer, Lawrence O Hall, and W Philip Kegelmeyer. 2002. SMOTE: synthetic minority over-sampling technique. Journal of artificial intelligence research 16 (2002), 321–357.
[11]
Syed Naser Daimi and Goutam Saha. 2014. Classification of emotions induced by music videos and correlation with participants’ rating. Expert Systems with Applications 41, 13 (2014), 6057–6065.
[12]
Paul Ekman. Accessed June 2023. Universal Emotions. https://www.paulekman.com/universal-emotions/.
[13]
Rishabh Gupta, Tiago H Falk, 2016. Relevance vector classifier decision fusion and EEG graph-theoretic features for automatic affective state characterization. Neurocomputing 174 (2016), 875–884.
[14]
Sepideh Hatamikia, Keivan Maghooli, and Ali Motie Nasrabadi. 2014. The emotion recognition system based on autoregressive model and sequential forward feature selection of electroencephalogram signals. Journal of medical signals and sensors 4, 3 (2014), 194.
[15]
Bo Hjorth. 1970. EEG analysis based on time domain properties. Electroencephalography and clinical neurophysiology 29, 3 (1970), 306–310.
[16]
Dan Huang, Suhua Zhang, and Yong Zhang. 2017. EEG-based emotion recognition using empirical wavelet transform. In 2017 4th international conference on systems and informatics (ICSAI). IEEE, 1444–1449.
[17]
Norden E Huang, Zheng Shen, Steven R Long, Manli C Wu, Hsing H Shih, Quanan Zheng, Nai-Chyuan Yen, Chi Chao Tung, and Henry H Liu. 1998. The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society of London. Series A: mathematical, physical and engineering sciences 454, 1971 (1998), 903–995.
[18]
Ashhadul Islam, Samir Brahim Belhaouari, Atiq Ur Rehman, and Halima Bensmail. 2022. KNNOR: An Oversampling Technique for Imbalanced Datasets. Appl. Soft Comput. 115, C (jan 2022), 18 pages. https://doi.org/10.1016/j.asoc.2021.108288
[19]
Robert Jenke, Angelika Peer, and Martin Buss. 2014. Feature extraction and selection for emotion recognition from EEG. IEEE Transactions on Affective computing 5, 3 (2014), 327–339.
[20]
Na Ji, Liang Ma, Hui Dong, and Xuejun Zhang. 2019. EEG signals feature extraction based on DWT and EMD combined with approximate entropy. Brain sciences 9, 8 (2019), 201.
[21]
Muhammad Kaleem, Behnaz Ghoraani, Aziz Guergachi, and Sridhar Krishnan. 2013. Pathological speech signal analysis and classification using empirical mode decomposition. Medical & biological engineering & computing 51 (2013), 811–821.
[22]
Keras. 2019. KERAS: The python deep learning library. https://keras.io/. Accessed: 2019.
[23]
Sander Koelstra, Christian Muhl, Mohammad Soleymani, Jong-Seok Lee, Ashkan Yazdani, Touradj Ebrahimi, Thierry Pun, Anton Nijholt, and Ioannis Patras. 2011. Deap: A database for emotion analysis; using physiological signals. IEEE transactions on affective computing 3, 1 (2011), 18–31.
[24]
Nitin Kumar, Kaushikee Khaund, and Shyamanta M Hazarika. 2016. Bispectral analysis of EEG for emotion recognition. Procedia Computer Science 84 (2016), 31–35.
[25]
Chao Li, Boyang Chen, Ziping Zhao, Nicholas Cummins, and Björn W Schuller. 2021. Hierarchical attention-based temporal convolutional networks for eeg-based emotion recognition. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 1240–1244.
[26]
Mi Li, Hongpei Xu, Xingwang Liu, and Shengfu Lu. 2018. Emotion recognition from multichannel EEG signals using K-nearest neighbor classification. Technology and health care 26, S1 (2018), 509–519.
[27]
Yisi Liu and Olga Sourina. 2013. Real-time fractal-based valence level recognition from EEG. In Transactions on computational science XVIII: special issue on Cyberworlds. Springer, 101–120.
[28]
Yuling Luo, Qiang Fu, Juntao Xie, Yunbai Qin, Guopei Wu, Junxiu Liu, Frank Jiang, Yi Cao, and Xuemei Ding. 2020. EEG-based emotion classification using spiking neural networks. IEEE Access 8 (2020), 46007–46016.
[29]
Zeynab Mohammadi, Javad Frounchi, and Mahmood Amiri. 2017. Wavelet-based emotion recognition system using EEG signal. Neural Computing and Applications 28 (2017), 1985–1990.
[30]
Daimi Syed Naser and Goutam Saha. 2021. Influence of music liking on EEG based emotion recognition. Biomedical Signal Processing and Control 64 (2021), 102251.
[31]
Hien M Nguyen, Eric W Cooper, and Katsuari Kamei. 2011. Borderline over-sampling for imbalanced data classification. International Journal of Knowledge Engineering and Soft Data Paradigms 3, 1 (2011), 4–21.
[32]
Rosalind W Picard. 2000. Affective computing. MIT press.
[33]
PyEMD. Accessed June 2023. PyEMD. https://pypi.org/project/pyemd/.
[34]
Viktor Rozgić, Shiv N Vitaladevuni, and Rohit Prasad. 2013. Robust EEG emotion classification using segment level decision fusion. In 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 1286–1290.
[35]
Jianzhuo Yan, Shangbin Chen, and Sinuo Deng. 2019. A EEG-based emotion recognition model with rhythm and time characteristics. Brain informatics 6, 1 (2019), 1–8.
[36]
Yongqiang Yin, Xiangwei Zheng, Bin Hu, Yuang Zhang, and Xinchun Cui. 2021. EEG emotion recognition using fusion model of graph convolutional neural networks and LSTM. Applied Soft Computing 100 (2021), 106954.
[37]
Yong Zhang, Xiaomin Ji, and Suhua Zhang. 2016. An approach to EEG-based emotion recognition using combined feature extraction method. Neuroscience letters 633 (2016), 152–157.

Index Terms

  1. Joint Class Learning with Self Similarity Projection for EEG Emotion Recognition
        Index terms have been assigned to the content through auto-classification.

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Other conferences
        CODS-COMAD '24: Proceedings of the 7th Joint International Conference on Data Science & Management of Data (11th ACM IKDD CODS and 29th COMAD)
        January 2024
        627 pages
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 04 January 2024

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. EEG
        2. Emotion
        3. Joint Class Learning
        4. Partial Least Square
        5. Self-similarity
        6. Subspace Learning

        Qualifiers

        • Short-paper
        • Research
        • Refereed limited

        Conference

        CODS-COMAD 2024

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • 0
          Total Citations
        • 62
          Total Downloads
        • Downloads (Last 12 months)54
        • Downloads (Last 6 weeks)4
        Reflects downloads up to 20 Feb 2025

        Other Metrics

        Citations

        View Options

        Login options

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        HTML Format

        View this article in HTML Format.

        HTML Format

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media