Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Intelligent Virtual Agents

IVA 2011: Intelligent Virtual Agents pp 216–227Cite as

Towards More Comprehensive Listening Behavior: Beyond the Bobble Head

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6895))

Abstract

Realizing effective listening behavior in virtual humans has become a key area of research, especially as research has sought to realize more complex social scenarios involving multiple participants and bystanders. A human listener’s nonverbal behavior is conditioned by a variety of factors, from current speaker’s behavior to the listener’s role and desire to participate in the conversation and unfolding comprehension of the speaker. Similarly, we seek to create virtual humans able to provide feedback based on their participatory goals and their partial understanding of, and reaction to, the relevance of what the speaker is saying as the speaker speaks. Based on a survey of existing psychological literature as well as recent technological advances in recognition and partial understanding of natural language, we describe a model of how to integrate these factors into a virtual human that behaves consistently with these goals. We then discuss how the model is implemented into a virtual human architecture and present an evaluation of behaviors used in the model.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Argyle, M., Cook, M.: Gaze and Mutual Gaze. Cambridge University Press, Cambridge (1976)

    Google Scholar 

  2. Argyle, M., Lalljee, M., Cook, M.: The effects of visibility on interaction in a dyad. Human Relations 21, 3–17 (1968)

    Article  Google Scholar 

  3. Bavelas, J.B., Coates, L., Johnson, T.: Listeners as co-narrators. Journal of Personality and Social Psychology 79, 941–952 (2000)

    Article  Google Scholar 

  4. Bevacqua, E., Pammi, S., Hyniewska, S.J., Schröder, M., Pelachaud, C.: Multimodal backchannels for embodied conversational agents. In: Safonova, A. (ed.) IVA 2010. LNCS, vol. 6356, pp. 194–200. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  5. Brunner, L.: Smiles can be back channels. Journal of Personality and Social Psychology 37(5), 728–734 (1979)

    Article  MathSciNet  Google Scholar 

  6. Callan, H., Chance, M., Pitcairn, T.: Attention and advertence in human groups. Social Science Inform. 12, 27–41 (1973)

    Article  Google Scholar 

  7. DeVault, D., Sagae, K., Traum, D.: Incremental interpretation and prediction of utterance meaning for interactive dialogue. Dialogue & Discourse 2(1), 143–170 (2011)

    Google Scholar 

  8. Dittmann, A., Llewellyn, L.: Relationship between vocalizations and head nods as listener responses. Journal of Personality and Social Psychology 9, 79–84 (1968)

    Article  Google Scholar 

  9. Ellsworth, P., Friedman, H., Perlick, D., Hoyt, M.: Some effects of gaze on subjects motivated to seek or to avoid social comparison. Journal of Experimental Social Pscyhology 14, 69–87 (1978)

    Article  Google Scholar 

  10. Friedman, H.S., Riggio, R.E.: Effect of individual differences in non-verbal expressiveness on transmission of emotion. Journal of Nonverbal Behavior 6(2), 96–104 (1981)

    Article  Google Scholar 

  11. Goffman, E.: Forms of Talk. University of Pennsylvania Press, Philadelphia (1981)

    Google Scholar 

  12. Goodwin, C.: Conversational organization: interaction between speakers and hearers. Academic Press, NY (1981)

    Google Scholar 

  13. Gratch, J., Okhmatovskaia, A., Lamothe, F., Marsella, S.C., Morales, M., van der Werf, R.J., Morency, L.-P.: Virtual rapport. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 14–27. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  14. Gu, E., Badler, N.I.: Visual attention and eye gaze during multiparty conversations with distractions. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 193–204. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  15. Hanks, W.F.: Language and Communicative Practices. Westview Press (1996)

    Google Scholar 

  16. Hartholt, A., Gratch, J., Weiss, L., The Gunslinger Team: At the virtual frontier: Introducing gunslinger, a multi-character, mixed-reality, story-driven experience. In: Ruttkay, Z., Kipp, M., Nijholt, A., Vilhjálmsson, H.H. (eds.) IVA 2009. LNCS, vol. 5773, pp. 500–501. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  17. Ikeda, K.: Triadic exchange pattern in multiparty communication: A case study of conversational narrative among friends. Language and culture 30(2), 53–65 (2009)

    Google Scholar 

  18. Jan, D., Traum, D.R.: Dynamic movement and positioning of embodied agents in multiparty conversations. In: Proc. of the 6th Int. Conference on Autonomous Agents and Multiagent Systems, pp. 59–66 (2007)

    Google Scholar 

  19. Jonsdottir, G.R., Gratch, J., Fast, E., Thórisson, K.R.: Fluid semantic back-channel feedback in dialogue: Challenges and progress. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 154–160. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  20. Kendon, A.: Conducting Interaction: Patterns of Behavior in Focused Encounters. Cambridge University Press, Cambridge (1990)

    Google Scholar 

  21. Kopp, S., Allwood, J., Grammer, K., Ahlsen, E., Stocksmeier, T.: Modeling embodied feedback with virtual humans. In: Wachsmuth, I., Knoblich, G. (eds.) ZiF Research Group International Workshop. LNCS (LNAI), vol. 4930, pp. 18–37. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  22. Lee, J., Marsella, S.: Predicting speaker head nods and the effects of affective information. IEEE Transactions on Multimedia 12(6), 552–562 (2010)

    Article  Google Scholar 

  23. Lee, J., Marsella, S.C.: Nonverbal behavior generator for embodied conversational agents. In: Gratch, J., Young, M., Aylett, R.S., Ballin, D., Olivier, P. (eds.) IVA 2006. LNCS (LNAI), vol. 4133, pp. 243–255. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  24. Maatman, R.M., Gratch, J., Marsella, S.C.: Natural behavior of a listening agent. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 25–36. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  25. Marsella, S., Gratch, J.: EMA: A process model of appraisal dynamics. Cognitive Systems Research 10(1), 70–90 (2009)

    Article  Google Scholar 

  26. Morency, L.P., de Kok, I., Gratch, J.: A probabilistic multimodal approach for predicting listener backchannels. In: Prendinger, H., Lester, J.C., Ishizuka, M. (eds.) IVA 2008. LNCS (LNAI), vol. 5208, pp. 70–84. Springer, Heidelberg (2008)

    Google Scholar 

  27. Vertegaa, R., der Veer, G.C.V., Vons, H.: Effects of gaze on multiparty mediated communication. In: Proc. of Graphics Interface, pp. 95–102 (2000)

    Google Scholar 

  28. Yngve, V.: On getting a word in edgewise. Papers from the 6th Regional Meeting, pp. 567–578 (April 1970)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Creative Technologies, University of Southern California, 12015 Waterfront Drive, Playa Vista, CA, USA, 90094

    Zhiyang Wang, Jina Lee & Stacy Marsella

Authors
  1. Zhiyang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jina Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stacy Marsella
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. School of Computer Science, Reykjavik University, Menntavegur 1, 101, Reykjavík, Iceland

    Hannes Högni Vilhjálmsson

  2. Bielefeld University, CITEC, P.O. Box 100131, 33501, Bielefeld, Germany

    Stefan Kopp

  3. Institute for Creative Technologies, University of Southern California, 12015 Waterfront Drive, 90094-2536, Playa Vista, CA, USA

    Stacy Marsella

  4. Reykjavik University, CADIA, Menntavegur 1, 101, Reykjavik, Iceland

    Kristinn R. Thórisson

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Wang, Z., Lee, J., Marsella, S. (2011). Towards More Comprehensive Listening Behavior: Beyond the Bobble Head. In: Vilhjálmsson, H.H., Kopp, S., Marsella, S., Thórisson, K.R. (eds) Intelligent Virtual Agents. IVA 2011. Lecture Notes in Computer Science(), vol 6895. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-23974-8_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-23974-8_24

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-23973-1

  • Online ISBN: 978-3-642-23974-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation