Abstract
Natural human-system interaction can facilitate the acceptance of technological systems. The ability of emotion recognition can hereby provide a significant contribution. Surprisingly, the field of emotion recognition is dominated by static machine learning approaches that do not account for the dynamics present in emotional processes. To overcome this limitation, we applied nonlinear autoregressive (NARX) models to predict emotion intensity from different physiological features extracted from galvanic skin response (GSR), heart rate (HR) and respiration (RSP) signals. NARX models consider the history of both the exogenous inputs (physiological signals) and the output (intensity). Emotions of different intensities were induced with images, while the physiological signals were recorded and the participants assessed their subjectively felt intensity in real-time. The intensity changes were analysed for three different emotion qualities: Happiness/Joy, Disappointment/Regret, Worry/Fear. While models were obtained for each individual, only the best set of parameters across individuals was considered for evaluation. Overall, it was found that the NARX models performed better than a sliding-window linear regression for all qualities. Furthermore, relevant features for the prediction of intensity and “ideal” delays between physiological features and the felt intensity to be captured by the model were identified. Overall, results underline the importance of considering dynamics in emotion recognition and prediction tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Ethical approval was obtained from the Research Ethics Committee of the Free University of Bozen-Bolzano.
- 2.
The running rate is computed with respect to a reference interval that is moving along with the evaluation window as time proceeds.
- 3.
For the purpose of obtaining the z-score necessary to compute r, the approximate method is used. However, the reported p-values are always obtained with the exact method adequate for a small sample size.
References
Alazrai, R., Lee, C.G.: An narx-based approach for human emotion identification. In: 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 4571–4576. IEEE (2012)
Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Therapy Exp. Psychiat. 25(1), 49–59 (1994)
Brave, S., Nass, C.: Emotion in human-computer interaction. In: The Human-Computer Interaction Handbook, pp. 103–118. CRC Press (2007)
Chang, C.Y., Chang, C.W., Lin, Y.M.: Application of support vector machine for emotion classification. In: 2012 Sixth International Conference on Genetic and Evolutionary Computing, pp. 249–252. IEEE (2012)
Derogatis, L.R.: Symptom checklist-90-revised, brief symptom inventory, and bsi-18. In: Handbook of Psychological Assessment in Primary Care Settings (2017)
Heylen, J., Verduyn, P., Van Mechelen, I., Ceulemans, E.: Variability in anger intensity profiles: structure and predictive basis. Cognit. Emotion 29(1), 168–177 (2015)
Jenke, R., Peer, A.: A cognitive architecture for modeling emotion dynamics: intensity estimation from physiological signals. Cognit. Syst. Res. 49, 128–141 (2018)
Kuppens, P., Verduyn, P.: Emotion dynamics. Curr. Opin. Psychol. 17, 22–26 (2017)
Lang, P.J.: International affective picture system (IAPS): affective ratings of pictures and instruction manual. Technical report (2005)
Li, M., Xu, H., Liu, X., Lu, S.: Emotion recognition from multichannel EEG signals using k-nearest neighbor classification. Technol. Health Care 26(S1), 509–519 (2018)
Lin, W., Li, C., Sun, S.: Deep convolutional neural network for emotion recognition using EEG and peripheral physiological signal. In: Zhao, Y., Kong, X., Taubman, D. (eds.) ICIG 2017. LNCS, vol. 10667, pp. 385–394. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-71589-6_33
Liu, S., Wang, X., Zhao, L., Zhao, J., Xin, Q., Wang, S.H.: Subject-independent emotion recognition of EEG signals based on dynamic empirical convolutional neural network. IEEE/ACM Trans. Comput. Biol. Bioinf. 18(5), 1710–1721 (2020)
Ljung, L.: Perspectives on system identification. Annu. Rev. Control 34(1), 1–12 (2010)
Mithbavkar, S.A., Shah, M.S.: Recognition of emotion through facial expressions using EMG signal. In: 2019 International Conference on Nascent Technologies in Engineering (ICNTE), pp. 1–6. IEEE (2019)
Moors, A., Ellsworth, P.C., Scherer, K.R., Frijda, N.H.: Appraisal theories of emotion: state of the art and future development. Emotion Rev. 5(2), 119–124 (2013)
Sacharin, V., Schlegel, K., Scherer, K.R.: Geneva emotion wheel rating study (2012)
Sammut, C., Webb, G.I. (eds.): Leave-One-Out Cross-Validation, pp. 600–601. Springer, Boston (2010). https://doi.org/10.1007/978-0-387-30164-8_469
Scherer, K.R.: The dynamic architecture of emotion: evidence for the component process model. Cognit. Emotion 23(7), 1307–1351 (2009)
Scherer, K.R., Schorr, A., Johnstone, T.: Appraisal Processes in Emotion: Theory, Methods, Research. Oxford University Press (2001)
Schneegans, S., Schöner, G.: Dynamic field theory as a framework for understanding embodied cognition. In: Handbook of Cognitive Science, pp. 241–271 (2008)
Shu, L., et al.: A review of emotion recognition using physiological signals. Sensors 18(7), 2074 (2018)
Sonnemans, J., Frijda, N.H.: The structure of subjective emotional intensity. Cognit. Emotion 8(4), 329–350 (1994)
Sprent, P., Smeeton, N.C.: Applied Nonparametric Statistical Methods. CRC Press (2016)
Torres-Valencia, C.A., Garcia-Arias, H.F., Lopez, M.A.A., Orozco-Gutiérrez, A.A.: Comparative analysis of physiological signals and electroencephalogram (EEG) for multimodal emotion recognition using generative models. In: 2014 XIX Symposium on Image, Signal Processing and Artificial Vision, pp. 1–5. IEEE (2014)
Valenza, G., Lanata, A., Scilingo, E.P.: The role of nonlinear dynamics in affective valence and arousal recognition. IEEE Trans. Affect. Comput. 3(2), 237–249 (2011)
Verduyn, P., Van Mechelen, I., Tuerlinckx, F., Meers, K., Van Coillie, H.: Intensity profiles of emotional experience over time. Cognit. Emotion 23(7), 1427–1443 (2009)
Acknowledgements
This work is supported in part by a research assistant position of the Free University of Bozen-Bolzano.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Barradas, I., Tschiesner, R., Peer, A. (2022). Towards a Dynamic Model for the Prediction of Emotion Intensity from Peripheral Physiological Signals. In: Kurosu, M., et al. HCI International 2022 - Late Breaking Papers. Multimodality in Advanced Interaction Environments. HCII 2022. Lecture Notes in Computer Science, vol 13519. Springer, Cham. https://doi.org/10.1007/978-3-031-17618-0_2
Download citation
DOI: https://doi.org/10.1007/978-3-031-17618-0_2
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-17617-3
Online ISBN: 978-3-031-17618-0
eBook Packages: Computer ScienceComputer Science (R0)