Abstract
Emotion recognition based on multimodal physiological signal has attracted a bunch of attention. Tensor learning helps to extract effective shared features from multi-modal high-dimensional data. However, in tensor decomposition, the determination of the core size has always been a difficult problem, resulting in the loss of effective feature information. In this paper, we propose a multi-core voting tensor learning method, namely MCVTL, for multimodal emotion analysis, which try to improve the results of emotion recognition by fusing multi-core information of various scales. Especially, through fusing the knowledge with 4 cores, the performance is improved by nearly 5% in valence and 6% in arousal compared with the single-core case. The empirical results demonstrate the effectiveness of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abtahi, F., Ro, T., Li, W., Zhu, Z.: Emotion analysis using audio/video, EMG and EEG: a dataset and comparison study. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 10–19. IEEE (2018)
Candra, H., et al.: Investigation of window size in classification of EEG-emotion signal with wavelet entropy and support vector machine. In: 2015 37th Annual international conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 7250–7253. IEEE (2015)
Chao, H., Dong, L., Liu, Y., Lu, B.: Emotion recognition from multiband EEG signals using CapsNet. Sensors 19(9), 2212 (2019)
Cong, F., Lin, Q.H., Kuang, L.D., Gong, X.F., Astikainen, P., Ristaniemi, T.: Tensor decomposition of EEG signals: a brief review. J. Neurosci. Methods 248, 59–69 (2015)
Daimi, S.N., Saha, G.: Classification of emotions induced by music videos and correlation with participants’ rating. Expert Syst. Appl. 41(13), 6057–6065 (2014)
De Lathauwer, L., De Moor, B., Vandewalle, J.: On the best rank-1 and rank-(R\(_{1}\), R\(_{2}\),..., R\(_N\)) approximation of higher-order tensors. SIAM J. Matrix Anal. Appl. 21(4), 1324–1342 (2000)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: 2004 IEEE International Joint Conference on Neural Networks (IEEE Cat. No. 04CH37541), vol. 2, pp. 985–990. IEEE (2004)
Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2011)
Lin, Y.P., Wang, C.H., Wu, T.L., Jeng, S.K., Chen, J.H.: EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. In: 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 489–492. IEEE (2009)
Mert, A., Akan, A.: Emotion recognition from EEG signals by using multivariate empirical mode decomposition. Pattern Anal. Appl. 21(1), 81–89 (2018). https://doi.org/10.1007/s10044-016-0567-6
Mill, A., Allik, J., Realo, A., Valk, R.: Age-related differences in emotion recognition ability: a cross-sectional study. Emotion 9(5), 619 (2009)
Mohammadi, Z., Frounchi, J., Amiri, M.: Wavelet-based emotion recognition system using EEG signal. Neural Comput. Appl. 28(8), 1985–1990 (2017). https://doi.org/10.1007/s00521-015-2149-8
Nikolova, D., Mihaylova, P., Manolova, A., Georgieva, P.: ECG-based human emotion recognition across multiple subjects. In: Poulkov, V. (ed.) FABULOUS 2019. LNICST, vol. 283, pp. 25–36. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-23976-3_3
Phan, A.H., Cichocki, A.: Tensor decompositions for feature extraction and classification of high dimensional datasets. Nonlinear Theory Appl. IEICE 1(1), 37–68 (2010)
Posner, J., Russell, J.A., Peterson, B.S.: The circumplex model of affect: an integrative approach to affective neuroscience, cognitive development, and psychopathology. Dev. Psychopathol. 17(3), 715 (2005)
Verma, G.K., Tiwary, U.S.: Multimodal fusion framework: a multiresolution approach for emotion classification and recognition from physiological signals. NeuroImage 102, 162–172 (2014)
Vogt, T., André, E.: Improving automatic emotion recognition from speech via gender differentiation. In: LREC, pp. 1123–1126 (2006)
Zheng, W.L., Zhu, J.Y., Lu, B.L.: Identifying stable patterns over time for emotion recognition from EEG. IEEE Trans. Affect. Comput. 10, 417–429 (2017)
Zhou, G., Cichocki, A., Xie, S.: Fast nonnegative matrix/tensor factorization based on low-rank approximation. IEEE Trans. Signal Process. 60(6), 2928–2940 (2012)
Zhuang, N., Zeng, Y., Tong, L., Zhang, C., Zhang, H., Yan, B.: Emotion recognition from EEG signals using multidimensional information in EMD domain. BioMed Res. Int. 2017, 1–9 (2017)
Acknowledgments
This work was supported by NSFC (61633010, 61671193, 61602140), National Key Research & Development Project (2017YFE0116800), Key Research & Development Project of Zhejiang Province (2020C04009, 2018C04012).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Xu, H., Tang, J., Zhang, J., Zhu, L. (2021). Emotion Recognition Using Multi-core Tensor Learning and Multimodal Physiological Signal. In: Wang, Y. (eds) Human Brain and Artificial Intelligence. HBAI 2021. Communications in Computer and Information Science, vol 1369. Springer, Singapore. https://doi.org/10.1007/978-981-16-1288-6_10
Download citation
DOI: https://doi.org/10.1007/978-981-16-1288-6_10
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-16-1287-9
Online ISBN: 978-981-16-1288-6
eBook Packages: Computer ScienceComputer Science (R0)