Abstract
The goal of this research was to develop a system that will automatically measure changes in the emotional state of a speaker, by analyzing his/her voice. Natural (non-acted) human speech of 77 (Dutch) speakers was collected and manually splitted into speech units. Three recordings per speaker were collected, in which he/she was in a positive, neutral and negative state. For each recording, the speakers rated 16 emotional states on a 10-point Likert Scale. The Random Forest algorithm was applied to 207 speech features that were extracted from recordings to qualify (classification) and quantify (regression) the changes in speaker’s emotional state. Results showed that predicting the direction of change of emotions and the change of intensity, measured by Mean Squared Error, can be done better than the baseline (the mean value of change). Moreover, it turned out that changes in negative emotions are more predictable than changes in positive emotions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Batliner, A., et al.: Whodunnit – Searching for the Most Important Speech Feature Types Signalling Emotion-related User States in Speech. Computer Speech and Language (2010), doi:10.1016/j.csl.2009.12.003
Breiman, L.: Random Forests. Machine Learning 45, 5–32 (2001)
Castellano, G., Kessous, L., Caridakis, G.: Emotion Recognition through Multiple Modalities: Face, Body Gesture, Speech. In: Peter, C., Beale, R. (eds.) Affect and Emotion in HCI. LNCS, vol. 4868, pp. 92–103. Springer, Heidelberg (2008)
Zhang, C., Wu, J., Xiao, X., Wang, Z.: Pronunciation variation modeling for Mandarin with accent. In: Proc. ICSLP 2006, Pittsburgh, USA, pp. 709–712 (2006)
Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, 2nd edn. Wiley (2000)
Frijda, N.H.: The laws of emotion. Lawrence Erlbaum Associates Publishers, Mahwah (2007)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning, ch. 15, 2nd edn. Springer, New York (2008)
Kurematsu, M., Amanuma, S., Hakura, J., Fujita, H.: An Extraction of Emotion in Human Speech Using Cluster Analysis and a Regression Tree. In: Fujita, H., Sasaki, J. (eds.) Proceedings of the 10th WSEAS International Conference on Applied Computer Science, pp. 346–350. World Scientific and Engineering Academy and Society (WSEAS), Stevens Point (2008)
Laukka, P., Neiberg, D., Forsell, M., Karlsson, I., Elenius, K.: Expression of Affect in Spontaneous Speech: Acoustic Correlates and Automatic Detection of Irritation and Resignation. Computer Speech and Language 25, 84–104 (2011)
Li, X., et al.: Stress and Emotion Classification Using Jitter and Shimmer Features. In: IEEE International Conference on Acoustics, Speech and Signal Processing, ICASSP 2007, pp. 1081–1084 (2007)
McIntyre, G., Göcke, R.: Towards affective sensing. In: Jacko, J.A. (ed.) HCI 2007, Part III. LNCS, vol. 4552, pp. 411–420. Springer, Heidelberg (2007)
Schölkopf, B., Smola, A.J.: Learning with Kernels. Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press (2001)
Tawari, A., Trivedi, M.: Speech Based Emotion Classification Framework for Driver Assistence System. In: Intelligent Vehicles Symposium (IV), June 21-24, pp. 174–178. IEEE (2010), doi:10.1109/IVS.2010.5547956
Vogt, T., André, E., Wagner, J.: Automatic Recognition of Emotions from Speech: A Review of the Literature and Recommendations for Practical Realisation. In: Peter, C., Beale, R. (eds.) Affect and Emotion in HCI. LNCS, vol. 4868, pp. 75–91. Springer, Heidelberg (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Kowalczyk, W., van der Wal, C.N. (2012). Detecting Changing Emotions in Natural Speech. In: Jiang, H., Ding, W., Ali, M., Wu, X. (eds) Advanced Research in Applied Artificial Intelligence. IEA/AIE 2012. Lecture Notes in Computer Science(), vol 7345. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31087-4_51
Download citation
DOI: https://doi.org/10.1007/978-3-642-31087-4_51
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31086-7
Online ISBN: 978-3-642-31087-4
eBook Packages: Computer ScienceComputer Science (R0)