Abstract
Modern research in the field of automatic emotion recognition systems are often dealing with acted affective databases. However, such data is far from the real-world problems. Human-computer interaction systems are extending their field of application and becoming a great part of human’s everyday life. Such systems are communicating with user through dialog and are supposed to define the current mood in order to adjust their behaviour. To increase the depth of emotion definition, multidimensional time-continuous labelling is used in this study instead of utterance-label categorical approach. Context-aware (long short-term memory recurrent neural network) and context-unaware (linear regression) models are contrasted and compared. Different meta-modelling techniques are applied to provide final labels for each dimension based on unimodal predictions. This study shows that context-awareness can lead to a significant increase in emotion recognition precision with time-continuous data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ringeval, F., Sonderegger, A., Sauer, J., Lalanne, D.: Introducing the RECOLA multimodal corpus of remote collaborative and affective interactions. In: Proceedings of IEEE on Face and Gestures 2013, 2nd International Workshop on Emotion Representation, Analysis and Synthesis in Continuous Time and Space (EmoSPACE), Shanghai, China (2013)
Lawrence, I., Kuei, L.: A concordance correlation coefficient to evaluate reproducibility. Biometrics 45, 255–268 (1989)
Denham, S.A.: Emotional Development in Young Children. Guilford Press, New York (1998)
Chronaki, G., et al.: The development of emotion recognition from facial expressions and non-linguistic vocalizations during childhood. Br. J. Dev. Psychol. 33(2), 218–236 (2015)
Nicholson, J., Kaxuhiko, T, Nakatsu, R.: Emotion recognition in speech using neural networks. In: Proceedings of 6th International Conference on Neural Information Processing (ICONIP 1999), vol. 2. IEEE (1999)
Markel, J.D., Gray, A.H.: Linear Prediction of Speech. Springer, New York (1976)
Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences, vol. 30. Springer, Heidelberg (1995)
Russell, J.A., Bachorowski, J.A., Fernández-Dols, J.M.: Facial and vocal expressions of emotion. Ann. Rev. Psychol. 54(1), 329–349 (2003)
Fragopanagos, N., Taylor, J.G.: Emotion recognition in human-computer interaction. Neural Netw. 18(4), 389–405 (2005)
Vijayan, A.E., Deepak, S., Sudheer, A.P.: EEG-based emotion recognition using statistical measures and auto-regressive modeling. In: IEEE International Conference on Computational Intelligence and Communication Technology (CICT). IEEE (2015)
Koelstra, S., et al.: DEAP: a database for emotion analysis; using physiological signals. IEEE Trans. Affect. Comput. 3(1), 18–31 (2012)
Kaya, H., Salah, A.: Combining modality-specific extreme learning machines for emotion recognition in the wild. J. Multimodal User Interfaces 10(2), 139–149 (2016)
Wöllmer, M., et al.: LSTM-modeling of continuous emotions in an audiovisual affect recognition framework. Image Vis. Comput. 31(2), 153–163 (2013)
Ringeval, F., Eyben, F., Kroupi, E., Yuce, A., Thiran, J.-P., Ebrahimi, T., Lalanne, D., Schuller, B.: Prediction of asynchronous dimensional emotion ratings from audio-visual and physiological data. Pattern Recogn. Lett. 66, 22–30 (2015)
Gunes, H., Pantic, M.: Automatic dimensional and continuous emotion recognition. Int. J. Synth. Emot. 1(1), 68–99 (2010)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Srivastava, N., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Nair, V., Hinton, G.E.: Rectified linear units improve restricted Boltzmann machines. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010) (2010)
Francois, C.: Keras (2015). https://github.com/fchollet/keras
Pedregosa, F., et al.: Scikit-learn: machine learning in python. JMLR 12, 2825–2830 (2011)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization (2014). arXiv preprint arXiv:1412.6980
Liu, D.C., Nocedal, J.: On the limited memory BFGS method for large scale optimization. Math. Program. 45(1), 503–528 (1989)
Breiman, L., et al.: Classification and regression trees. Wadsworth & Brooks, Monterey (1984)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Acknowledgments
The work presented in this paper was partially supported by the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” which is funded by the German Research Foundation (DFG).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Fedotov, D., Sidorov, M., Minker, W. (2017). Context-Awared Models in Time-Continuous Multidimensional Affect Recognition. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2017. Lecture Notes in Computer Science(), vol 10459. Springer, Cham. https://doi.org/10.1007/978-3-319-66471-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-66471-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66470-5
Online ISBN: 978-3-319-66471-2
eBook Packages: Computer ScienceComputer Science (R0)