Skip to main content
SpringerLink
Log in
Book cover

International Conference on Hybrid Artificial Intelligence Systems

HAIS 2015: Hybrid Artificial Intelligent Systems pp 357–368Cite as

Modeling Users Emotional State for an Enhanced Human-Machine Interaction

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9121))

Abstract

Spoken conversational agents have been proposed to enable a more natural and intuitive interaction with the environment and human-computer interfaces. In this paper, we propose a framework to model the user’s emotional state during the dialog and adapt the dialog model dynamically, thus developing more efficient, adapted, and usable conversational agents. We have evaluated our proposal developing a user-adapted agent that facilitates touristic information, and provide a detailed discussion of the positive influence of our proposal in the success of the interaction, the information and services provided, as well as the perceived quality.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Pieraccini, R., Rabiner, L.: The Voice in the Machine: Building Computers That Understand Speech. MIT Press, Cambridge (2012)

    Google Scholar 

  2. Osland, P., Viken, B., Solsvik, F., Nygreen, G., Wedvik, J., Myklbust, S.: Enabling context-aware applications. In: Proceedings of the International Conference on Convergence in Services, Media and Networks (ICIN 2006), pp. 1–6 (2006)

    Google Scholar 

  3. Strauss, P., Minker, W.: Proactive Spoken Dialogue Interaction in Multi-Party Environments. Springer, US (2010)

    Book  MATH  Google Scholar 

  4. Kartakis, S.: A design-and-play approach to accesible user interface development in ambient intelligence environments. J. Comput. Ind. 61(4), 318–328 (2010). Elsevier

    Google Scholar 

  5. Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun. 53(9–10), 1062–1087 (2011). Elsevier

    Google Scholar 

  6. Batliner, A., Burkhardt, F., van Ballegooy, M., Nöth, E.: A taxonomy of applications that utilize emotional awareness. In: Proceedings of the 1st International Language Technologies Conference (IS-LTC 2006), pp. 246–250 (2006)

    Google Scholar 

  7. Bickmore, T., Giorgino, T.: Some novel aspects of health communication from a dialogue systems perspective. In: Proceedings of AAAI Fall Symposium on Dialogue Systems for Health Communication, pp. 275–291 (2004)

    Google Scholar 

  8. Litman, D., Forbes-Riley, K.: Recognizing student emotions and attitudes on the basis of utterances in spoken tutoring dialogues with both human and computer tutors. Speech Commun. 48(5), 559–590 (2006). Elsevier

    Google Scholar 

  9. Khalifa, O., Ahmad, Z., Gunawan, T.: SMaTTS: standard malay text to speech system. Int. J. Comput. Sci. 2(4), 285–293 (2007). JARCS

    Google Scholar 

  10. Acosta, J., Ward, N.: Responding to user emotional state by adding emotional coloring to utterances. In: Proceeding of 10th Annual Conference of the International Speech Communication Association (Interspeech 2009), pp. 1587–1590 (2009)

    Google Scholar 

  11. Boril, H., Hansen, J.: Unsupervised equalization of Lombard effect for speech recognition in noisy adverse environments. IEEE Trans. Audio Speech Lang. Process. 28(6), 1379–1393 (2010). IEEE

    Google Scholar 

  12. Bosma, W., Andre, E.: Exploiting emotions to disambiguate dialogue acts. In: Proceedings of 9th International Conference on Intelligent User Interface, pp. 85–92 (2004)

    Google Scholar 

  13. Wilks, Y., Catizone, R., Worgan, S., Turunen, M.: Some background on dialogue management and conversational speech for dialogue systems. Comput. Speech Lang. 25(2), 128–139 (2011). Elsevier

    Google Scholar 

  14. Riccardi, G., Hakkani-Tür, D.: Grounding emotions in human-machine conversational systems. In: Maybury, M., Stock, O., Wahlster, W. (eds.) INTETAIN 2005. LNCS (LNAI), vol. 3814, pp. 144–154. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  15. Marreiros, G., Santos, R., Ramos, C., Neves, J.: Context-aware emotion-based model for group decision making. IEEE Intell. Syst. 25(2), 31–39 (2010). IEEE

    Google Scholar 

  16. Santos, R., Marreiros, G., Ramos, C., Neves, J., Bulas-Cruz, J.: Personality, emotion, and mood in agent-based group decision making. IEEE Intell. Syst. 26(6), 58–66 (2011). IEEE

    Google Scholar 

  17. Pittermann, J., Pittermann, A., Minker, W.: Emotion recognition and adaptation in spoken dialogue systems. Int. J. Speech Technol. 13, 49–60 (2010). Springer

    Google Scholar 

  18. Bui, T., Poel, M., Nijholt, A., Zwiers, J.: A tractable hybrid DDN-POMDP approach to affective dialogue modeling for probabilistic frame-based dialogue systems. Nat. Lang. Eng. 15(2), 273–307 (2009). Cambridge University Press

    Google Scholar 

  19. Williams, J., Young, S.: Partially observable Markov decision processes for spoken dialogue systems. Comput. Speech Lang. 21, 393–422 (2007). Elsevier

    Google Scholar 

  20. Callejas, Z., López-Cózar, R.: Influence of contextual information in emotion annotation for spoken dialogue systems. Speech Commun. 50(5), 416–433 (2008). Elsevier

    Google Scholar 

  21. Witten, I., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques. Morgan Kaufmann, San Francisco (2005)

    Google Scholar 

  22. Hansen, J.: Analysis and compensation of speech under stress and noise for environmental robustness in speech recognition. Speech Commun. 20(2), 151–170 (1996). Elsevier

    Google Scholar 

  23. Ververidis, D., Kotropoulos, C.: Emotional speech recognition: resources, features and methods. Speech Commun. 48, 1162–1181 (2006). Elsevier

    Google Scholar 

  24. Morrison, D., Wang, R., Silva, L.: Ensemble methods for spoken emotion recognition in call-centers. Speech Commun. 49(2), 98–112 (2007). Elsevier

    Google Scholar 

  25. Batliner, A., Steidl, S., Schuller, B., Seppi, D., Vogt, T., Wagner, J., Devillers, L., Vidrascu, L., Aharonson, V., Kessous, L., Amir, N.: Whodunnit - searching for the most important feature types signalling emotion-related user states in speech. Comput. Speech Lang. 25(1), 4–28 (2011). Elsevier

    Google Scholar 

  26. Burkhardt, F., van Ballegooy, M., Engelbrecht, K., Polzehl, T., Stegmann, J.: Emotion detection in dialog systems - usecases, strategies and challenges. In: Proceedings of International Conference on Affective Computing and Intelligent Interaction (ACII 2009), pp. 1–6 (2009)

    Google Scholar 

  27. Griol, D., García-Jiménez, M.: Development of interactive virtual voice portals to provide municipal information. Adv. Intell. Soft Comput. 151, 161–172 (2012). Springer

    Google Scholar 

  28. Will, T.: A Simple Guide to IBM SPSS: For Version 20.0. Akademiker Verlag (2012)

    Google Scholar 

  29. Ai, H., Raux, A., Bohus, D., Eskenazi, M., Litman, D.: Comparing spoken dialog corpora collected with recruited subjects versus real users. In: Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue, pp. 124–131 (2007)

    Google Scholar 

  30. Schatzmann, J., Georgila, K., Young, S.: Quantitative evaluation of user simulation techniques for spoken dialogue systems. In: Proceedings of the 6th SIGdial Workshop on Discourse and Dialogue, pp. 45–54 (2005)

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by Projects MINECO TEC2012-37832-C02-01, CICYT TEC2011-28626-C02-02, CAM CONTEXTS (S2009/TIC-1485).

Author information

Authors and Affiliations

  1. Computer Science Department, Carlos III University of Madrid, Avda. de la Universidad, 30, 28911, Leganés, Spain

    David Griol & José Manuel Molina

Authors
  1. David Griol
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. José Manuel Molina
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to David Griol .

Editor information

Editors and Affiliations

  1. University of Deusto, Bilbao, Spain

    Enrique Onieva

  2. University of Deusto, Bilbao, Spain

    Igor Santos

  3. University of Deusto, Bilbao, Spain

    Eneko Osaba

  4. Universidad de Salamanca, Salamanca, Spain

    Héctor Quintián

  5. Universidad de Salamanca, Salamanca, Spain

    Emilio Corchado

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Griol, D., Molina, J.M. (2015). Modeling Users Emotional State for an Enhanced Human-Machine Interaction. In: Onieva, E., Santos, I., Osaba, E., Quintián, H., Corchado, E. (eds) Hybrid Artificial Intelligent Systems. HAIS 2015. Lecture Notes in Computer Science(), vol 9121. Springer, Cham. https://doi.org/10.1007/978-3-319-19644-2_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-19644-2_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-19643-5

  • Online ISBN: 978-3-319-19644-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation