Skip to main content
SpringerLink
Log in
Book cover

Agent and Multi-agent Technology for Internet and Enterprise Systems pp 323–345Cite as

Web-Based Affective Human-Agent Interaction Generation

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 289))

Abstract

Virtual agents, as a promising technology for human-computer interaction, have become focus of research community in resent years. They serve as communicative fellows in a variety of applications. Employing virtual agents to realize human-computer communication on the web is a promising way to make the interaction attractive. In order to make use of intelligent interaction in the web by virtual agents, an important issue is that we should have a scripting language, which is easy to be used by authors. In this chapter, we discuss our research on the Multimodal Interaction Markup Language (MIML), which is a powerful and easy-to-use XML-based language. Different from the related languages in existence, MIML can script not only the presentations of virtual agents, but also their affective capability. We will describe the architecture of MIML, the facial expression recognition, speech emotion recognition, emotional speech synthesis ActiveX controllers and illustrate one scenario that instantiates the affective web-based human-agent interaction scripted by MIML. With the MIML we designed, web-based affective interaction can be described and generated easily.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Allen, J., Byron, D., Dzikovska, M., Ferguson, G., Galescu, L., Stent, A.: Towards Conversational Human-Computer Interaction. AI Magazine 22(4), 27–38 (2001)

    Google Scholar 

  2. Cassell, J., Sullivan, J., Prevost, S., Churchill, E.: Embodied Conversational Agents. The MIT Press, Cambridge (2000)

    Google Scholar 

  3. Picard, R.: Affective Computing. The MIT Press, Cambridge (2000)

    Google Scholar 

  4. Preece, J., Rogers, Y., Sharp, H.: Interaction Design, Beyond Human-Computer Interaction. John Wiley&Sons, Inc., Chichester (2002)

    Google Scholar 

  5. MIT Media Lab, http://affect.media.mit.edu/

  6. Toda, M.: Basic Structure of the Urge Operations, in the urge theory of emotion and cognition. SCCS Technical report, Chuyko University, Nagoya (1994)

    Google Scholar 

  7. Minsky, M.: The Society of Mind. Simon and Schuster, New York (1986)

    Google Scholar 

  8. Prendinger, H., Descamps, S., Ishizuka, M.: MPML: A Markup Language for Controlling the Behavior of Life-like Characters. Journal of Visual Languages and Computing 15(2), 183–203 (2004)

    Article  Google Scholar 

  9. Prendinger, H., Ishizuka, M.: Life-Like Characters-Tools, Affective Functions and Applications. Cognitive Technologies Series. Springer, Heidelberg (2004)

    Google Scholar 

  10. Marriott, A., Stallo, J.: VHML - Uncertainties and problems, A discussion. In: Proc. AAMAS 2002 Workshop on ECA-Let’s Specify and Evaluate Them, Bologna, Italy (2002)

    Google Scholar 

  11. VHML Home Page, http://www.vhml.org/

  12. Cassell, J., Vilhjalmsson, H., Bickmore, T.: BEAT: The Behavior Expression Animation Toolkit. In: Proc. SIGGRAPH 2001, Los Angeles, USA, pp. 477–486 (2001)

    Google Scholar 

  13. DeCarolis, B., Caroglio, V., Bilvi, M., Pelachaud, C.: APML: a Mark-up Language for Believable Behavior Generation. In: Proc. AAMAS 2002 Workshop on ECA-Let’s Specify and Evaluate Them, Bologna, Italy (2002)

    Google Scholar 

  14. Kopp, S., Wachsmuth, I.: Synthesizing Multimodal Utterances for Conversational Agents. Computer Animation and Virtual Worlds 15(1), 39–52 (2004)

    Article  Google Scholar 

  15. Kopp, S., Krenn, B., Marsella, S., Marshall, A., Pelachaud, C., Pirker, H.: Towards a Common Framework for Multimodal Generation: the Behavior Markup Language. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 205–217. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  16. Heylen, D., Kopp, S., Mareslla, S., Pelachaud, C., Vilhjalmsson, H.: The Next Step towards a Function Markup Language. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 270–280. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  17. Heylen, D., Maat, M.: A Linguistic View on Functional Markup Language. In: Proc. AAMAS 2008 Workshop on Functional Markup Language, Estoril (2008)

    Google Scholar 

  18. Kreen, B., Sieber, G.: Functional Markup for behavior Planning. Theory and Practice. In: Proc. AAMAS 2008, Estoril (2008)

    Google Scholar 

  19. Prendinger, H., Ishizuka, M.: Scream: Scripting Emotion-based Agent Minds. In: Proceeding of AAMAS 2002 Workshop on ECA-Let’s Specify and Evaluate Them, Italy (2002)

    Google Scholar 

  20. Prendinger, H., Descamps, S., Ishizuka, M.: Scripting Affective Communication with Virtual Characters in Web-based Interaction System. Applied Artificial Intelligence (2002)

    Google Scholar 

  21. Prendinger, H., Ishizuka, M.: Virtual Characters Tools, Affective Functions and Applications. Cognitive Technologies Series. Springer, Heidelberg (2004)

    Google Scholar 

  22. Prendinger, H., Ishizuka, M.: The Empathic Companion: a Character-based Interface that Addresses User’s Affective States. Journal of Applied Artificial Intelligence 19(3-4), 267–285 (2005)

    Article  Google Scholar 

  23. MPML Home Page, http://www.miv.t.u-tokyo.ac.jp/MPML/mpml.html

  24. Nischt, M., Prendinger, H., Ishizuka, M.: MPML3D: A Reactive Framework for the Multimodal Presentation Markup Language. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 218–229. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  25. Ullrich, S., Bruegmann, K., Prendinger, H., Ishizuka, M.: Extending MPML3D to Second Life. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 281–288. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  26. TVML Home Page, http://www.nhk.or.jp/strl/tvml/

  27. Badler, N.: Parameterized Action Representation for Virtual Human Agents. The MIT Press, Cambridge (2000)

    Google Scholar 

  28. PAR Home Page, http://hms.upenn.edu/software/par/

  29. Huang, Z., Eliebs, A.: STEP: a Scripting Language for Embodied Agent. In: Proc. of PRCAI 2002 Workshop on Virtual Animated Agent - Tools, Affective Functions and Applications, Tokyo (2002)

    Google Scholar 

  30. Microsoft Agent Home Page, http://www.microsoft.com/msagent

  31. Microsoft, Developing for Microsoft Agent. The Microsoft Press (1998)

    Google Scholar 

  32. SMIL Home Page, http://www.w3.org/AudioVideo/

  33. Huang, T., Chen, L., Tao, H.: Bimodal Emotion Recognition by Man and Machine. In: ATR Workshop on Virtual Communication Environments (1998)

    Google Scholar 

  34. DeSilva, L., Miyasato, T., Nakatsu, R.: Facial Emotion Recognition Using Multimodal Information. In: Han, Y., Quing, S. (eds.) ICICS 1997. LNCS, vol. 1334, pp. 397–401. Springer, Heidelberg (1997)

    Google Scholar 

  35. Mao, X., Xue, Y.L.: Beihang University Facial Expression Database and Multiple Facial Expression Recognition. In: Proc. of ICMLC 2006, pp. 369–372 (2006)

    Google Scholar 

  36. Viola, P.: Rapid Object Detection Using a Boosted Cascade of Simple Features. In: Proc. CVPR 2001, pp. 511–518 (2001)

    Google Scholar 

  37. Mao, X., Zhang, B., Luo, Y.: Speech Emotion Recognition Based on a Hybrid of HMM/ANN. In: Proc. WSEAS 2007, pp. 181–184 (2007)

    Google Scholar 

  38. Moulin, H.: Axioms of cooperative decision making. Cambridge University Press, Cambridge (1988)

    MATH  Google Scholar 

  39. Mao, X., Li, Z., Bao, H.Y.: Describing and Generating Web-based Affective Human-agent Interaction. In: Lovrek, I., Howlett, R.J., Jain, L.C. (eds.) KES 2008, Part I. LNCS (LNAI), vol. 5177, pp. 625–632. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, Beihang University, Beijing, China, 100191

    Xia Mao & Zheng Li

Authors
  1. Xia Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1.   , Department of Computer and Systems Sciences, Forum 100, SE-164 40, Kista, Sweden

    Anne Hãkansson

  2. Division of Computer and Information Sciences, Franklin University , 43224, Columbus Ohio, USA

    Ronald Hartung

  3. Institute of Computer Science, Wroclaw University of Technology , Str.Wyb.Wypsianskiego 27, 50-370, Wroclaw, Poland

    Ngoc Thanh Nguyen

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Mao, X., Li, Z. (2010). Web-Based Affective Human-Agent Interaction Generation. In: Hãkansson, A., Hartung, R., Nguyen, N.T. (eds) Agent and Multi-agent Technology for Internet and Enterprise Systems. Studies in Computational Intelligence, vol 289. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13526-2_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-13526-2_15

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-13525-5

  • Online ISBN: 978-3-642-13526-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation