Skip to main content
SpringerLink
Log in

Cross-Subject EEG-Based Emotion Recognition with Deep Domain Confusion

  • Conference paper
  • First Online:
Book cover Intelligent Robotics and Applications (ICIRA 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11740))

Included in the following conference series:

Abstract

At present, the method of emotion recognition based on Electroencephalogram (EEG) signals has received extensive attention. EEG signals have the characteristics of non-linear, non-stationary and low spatial resolution. There are great differences between EEG signals collected from different subjects as well as the same subjects from different experimental sessions. Therefore, it’s difficult for traditional emotion recognition methods to achieve high recognition accuracy. To tackle this problem, this paper proposes a cross-subject emotion recognition method based on convolutional neural network (CNN) and deep domain confusion (DDC). Firstly, the Electrodes-frequency Distribution Maps (EFDMs) is constructed from EEG signals, and the residual blocks based deep CNN is used to automatically extract the features related emotion recognition from the EFDMs. Then, the difference of the feature distribution between source and target domain are narrowed by the DDC. Finally, the EEG emotion recognition task is realized with EFDMs and CNN. On SEED, we set up two experiments, the proposed method achieved an average accuracy of 90.59% and 82.16%/4.43% for mean accuracy and standard deviation under conventional and cross-subject experimental protocols, respectively. Finally, this paper uses the gradient-weighted class activation mapping (Grad-CAM) to get a glimpse of what features the CNN has learned during the training from EFDMs, and obtained the conclusion that the high frequency EEG signals are more favorable for emotion recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Mühl, C., Jeunet, C., Lotte, F.: EEG-based workload estimation across affective contexts. Front. Neurosci. 8, 114 (2014)

    Google Scholar 

  2. Al-Kaysi, A.M., Al-Ani, A., Loo, C.K., et al.: Predicting tDCS treatment outcomes of patients with major depressive disorder using automated EEG classification. J. Affect. Disord. 208, 597–603 (2017)

    Article  Google Scholar 

  3. Fan, Y., Lu, X., Li, D., et al.: Video-based emotion recognition using CNN-RNN and C3D hybrid networks. In: Proceedings of the 18th ACM International Conference on Multimodal Interaction, pp. 445–450. ACM (2016)

    Google Scholar 

  4. Yan, J., Zheng, W., Cui, Z., et al.: Multi-cue fusion for emotion recognition in the wild. Neurocomputing 309, 27–35 (2018)

    Article  Google Scholar 

  5. Zhang, J., Chen, M., Zhao, S., et al.: ReliefF-based EEG sensor selection methods for emotion recognition. Sensors 16(10), 1558 (2016)

    Article  Google Scholar 

  6. Chen, J., Hu, B., Wang, Y., et al.: A three-stage decision framework for multi-subject emotion recognition using physiological signals. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 470–474. IEEE (2016)

    Google Scholar 

  7. Kollia, V., Elibol, O.H.: Distributed processing of biosignal-database for emotion recognition with mahout. arXiv preprint arXiv:1609.02631 (2016)

  8. Zheng, W.L., Guo, H.T., Lu, B.L.: Revealing critical channels and frequency bands for emotion recognition from EEG with deep belief network. In: 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER), pp. 154–157. IEEE (2015)

    Google Scholar 

  9. Zheng, W.L., Zhu, J.Y., Peng, Y., et al.: EEG-based emotion classification using deep belief networks. In: 2014 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2014)

    Google Scholar 

  10. Zheng, W.L., Liu, W., Lu, Y., et al.: EmotionMeter: a multimodal framework for recognizing human emotions. IEEE Trans. Cybern. 99, 1–13 (2018)

    Google Scholar 

  11. Pan, S.J., Tsang, I.W., Kwok, J.T., et al.: Domain adaptation via transfer component analysis. IEEE Trans. Neural Netw. 22(2), 199–210 (2011)

    Article  Google Scholar 

  12. Borgwardt, K.M., Gretton, A., Rasch, M.J., et al.: Integrating structured biological data by kernel maximum mean discrepancy. Bioinformatics 22(14), e49–e57 (2006)

    Article  Google Scholar 

  13. Fernando, B., Habrard, A., Sebban, M., et al.: Subspace alignment for domain adaptation. arXiv preprint arXiv:1409.5241 (2014)

  14. Sun, B., Saenko, K.: Subspace distribution alignment for unsupervised domain adaptation. In: BMVC, vol. 4, pp. 24.1–24.10 (2015)

    Google Scholar 

  15. Yin, Z., Wang, Y., Liu, L., et al.: Cross-subject EEG feature selection for emotion recognition using transfer recursive feature elimination. Front. Neurorobotics 11, 19 (2017)

    Article  Google Scholar 

  16. Yin, Z., Zhang, J.: Cross-session classification of mental workload levels using EEG and an adaptive deep learning model. Biomed. Signal Process. Control 33, 30–47 (2017)

    Article  Google Scholar 

  17. Zheng, W.L., Lu, B.L.: Personalizing EEG-based affective models with transfer learning. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence, pp. 2732–2738. AAAI Press (2016)

    Google Scholar 

  18. Chai, X., Wang, Q., Zhao, Y., et al.: Unsupervised domain adaptation techniques based on auto-encoder for non-stationary EEG-based emotion recognition. Comput. Biol. Med. 79, 205–214 (2016)

    Article  Google Scholar 

  19. Chai, X., Wang, Q., Zhao, Y., et al.: A fast, efficient domain adaptation technique for cross-domain electroencephalography (EEG)-based emotion recognition. Sensors 17(5), 1014 (2017)

    Article  Google Scholar 

  20. Tzeng, E., Hoffman, J., Zhang, N., et al.: Deep domain confusion: maximizing for domain invariance. arXiv preprint arXiv:1412.3474 (2014)

  21. Li, M., Chen, W., Zhang, T.: Classification of epilepsy EEG signals using DWT-based envelope analysis and neural network ensemble. Biomed. Signal Process. Control 31, 357–365 (2017)

    Article  Google Scholar 

  22. He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  23. Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. arXiv preprint arXiv:1502.03167 (2015)

  24. Zheng, W.L., Lu, B.L.: Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans. Auton. Ment. Dev. 7(3), 162–175 (2015)

    Article  Google Scholar 

  25. Thompson, B.: Canonical correlation analysis. In: Encyclopedia of Statistics in Behavioral Science (2005)

    Google Scholar 

  26. Zheng, W.: Multichannel EEG-based emotion recognition via group sparse canonical correlation analysis. IEEE Trans. Cogn. Dev. Syst. 9(3), 281–290 (2017)

    Article  Google Scholar 

  27. Liu, W, Zheng, W.L., Lu, B.L.: Multimodal emotion recognition using multimodal deep learning. arXiv preprint arXiv:1602.08225 (2016)

  28. Li, Y., Zheng, W., Cui, Z., Zhou, X.: A novel graph regularized sparse linear discriminant analysis model for EEG emotion recognition. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9950, pp. 175–182. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46681-1_21

    Chapter  Google Scholar 

  29. Ganin, Y., Ustinova, E., Ajakan, H., et al.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17(1), 2096–2130 (2016)

    MathSciNet  MATH  Google Scholar 

  30. Lan, Z., Sourina, O., Wang, L., et al.: Domain adaptation techniques for EEG-based emotion recognition: a comparative study on two public datasets. IEEE Trans. Cogn. Dev. Syst. 11(1), 85–94 (2019)

    Article  Google Scholar 

  31. Yan, K., Kou, L., Zhang, D.: Learning domain-invariant subspace using domain features and independence maximization. IEEE Trans. Cybern. 48(1), 288–299 (2018)

    Article  Google Scholar 

  32. Long, M., Ding, G., Wang, J., et al.: Transfer sparse coding for robust image representation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 407–414 (2013)

    Google Scholar 

  33. Long, M., Wang, J., Ding, G., et al.: Transfer joint matching for unsupervised domain adaptation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1410–1417 (2014)

    Google Scholar 

  34. Selvaraju, R.R, Cogswell, M., Das, A., et al.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Robot Science and Engineering, Northeastern University, Shenyang, Liaoning, China

    Weiwei Zhang, Fei Wang, Yang Jiang & Shichao Wu

  2. College of Information Science and Engineering, Northeastern University, Shenyang, Liaoning, China

    Zongfeng Xu & Yahui Zhang

Authors
  1. Weiwei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zongfeng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shichao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yahui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Wang .

Editor information

Editors and Affiliations

  1. Shenyang Institute of Automation, Shenyang, China

    Haibin Yu

  2. Shenyang Institute of Automation, Shenyang, China

    Jinguo Liu

  3. Shenyang Institute of Automation, Shenyang, China

    Lianqing Liu

  4. University of Portsmouth, Portsmouth, UK

    Zhaojie Ju

  5. Shenyang Institute of Automation, Shenyang, China

    Yuwang Liu

  6. University of Portsmouth, Portsmouth, UK

    Dalin Zhou

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhang, W., Wang, F., Jiang, Y., Xu, Z., Wu, S., Zhang, Y. (2019). Cross-Subject EEG-Based Emotion Recognition with Deep Domain Confusion. In: Yu, H., Liu, J., Liu, L., Ju, Z., Liu, Y., Zhou, D. (eds) Intelligent Robotics and Applications. ICIRA 2019. Lecture Notes in Computer Science(), vol 11740. Springer, Cham. https://doi.org/10.1007/978-3-030-27526-6_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-27526-6_49

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-27525-9

  • Online ISBN: 978-3-030-27526-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation