Skip to main content
SpringerLink
Log in

The Effects of Body Gestures and Gender on Viewer’s Perception of Animated Pedagogical Agent’s Emotions

  • Conference paper
  • First Online:
Human-Computer Interaction. Multimodal and Natural Interaction (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12182))

Included in the following conference series:

Abstract

The goal of this research is to develop Animated Pedagogical Agents (APA) that can convey clearly perceivable emotions through speech, facial expressions and body gestures. In particular, the two studies reported in the paper investigated the extent to which modifications to the range of movement of 3 beat gestures, e.g., both arms synchronous outward gesture, both arms synchronous forward gesture, and upper body lean, and the agent‘s gender have significant effects on viewer’s perception of the agent’s emotion in terms of valence and arousal. For each gesture the range of movement was varied at 2 discrete levels. The stimuli of the studies were two sets of 12-s animation clips generated using fractional factorial designs; in each clip an animated agent who speaks and gestures, gives a lecture segment on binomial probability. 50% of the clips featured a female agent and 50% of the clips featured a male agent. In the first study, which used a within-subject design and metric conjoint analysis, 120 subjects were asked to watch 8 stimuli clips and rank them according to perceived valence and arousal (from highest to lowest). In the second study, which used a between-subject design, 300 participants were assigned to two groups of 150 subjects each. One group watched 8 clips featuring the male agent and one group watched 8 clips featuring the female agent. Each participant was asked to rate perceived valence and arousal for each clip using a 7-point Likert scale. Results from the two studies suggest that the more open and forward the gestures the agent makes, the higher the perceived valence and arousal. Surprisingly, agents who lean their body forward more are not perceived as having higher arousal and valence. Findings also show that female agents’ emotions are perceived as having higher arousal and more positive valence that male agents’ emotions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Schroeder, N., Adesope, O.O., Barouch Gilbert, R.: How effective are pedagogical agents for learning? A meta-analytic review. J. Educ. Comput. Res. 49(1), 1–39 (2013)

    Article  Google Scholar 

  2. Kim, Y., Baylor, A.L.: Pedagogical agents as social models to influence learner attitudes. Educ. Technol. 47(01), 23–28 (2007)

    Google Scholar 

  3. Zhou, W., Cheng, J., Lei, X., Benes, B., Adamo, N.: Deep learning-based emotion recognition from real-time videos. In: Proceedings of HCI International 2020, Copenhagen, Denmark (2020, in press)

    Google Scholar 

  4. Kim, Y., Lim, J.: Gendered socialization with an embodied agent: creating a social and affable mathematics learning environment for middle-grade females. J. Educ. Psychol. 105(4), 1164–1174 (2013)

    Article  Google Scholar 

  5. Ekman, P., Friesen, W.: The repertoire of nonverbal behavior: categories, origins, usage, and coding. Semiotica 1(1), 49–98 (1969)

    Article  Google Scholar 

  6. Lhommet, M., Marsella, S.: Expressing emotion through posture and gesture. In: Calvo, R., D’Mello, S., Gratch, J., Kappas, A. (eds.) The Oxford Handbook of Affective Computing, pp. 273–285. Oxford University Press, Oxford (2015)

    Google Scholar 

  7. Russell, J.A.: Core affect and the psychological construct of emotion. Psychol. Rev. 110, 145–172 (2003)

    Article  Google Scholar 

  8. Johnson, W.L., Lester, J.C.: Face-to-face interaction with pedagogical agents, twenty years later. Int. J. Artif. Intell. Educ. 26(1), 25–36 (2016)

    Article  Google Scholar 

  9. Martha, A.S.D., Santoso, H.B.: The design and impact of the pedagogical agent: a systematic literature review. J. Educ. Online, 16(1) (2019)

    Google Scholar 

  10. Annetta, L.A., Holmes, S.: Creating presence and community in a synchronous virtual learning environment using avatars. Int. J. Instr. Technol. Distance Learn. 3, 27–43 (2006)

    Google Scholar 

  11. Alseid, M., Rigas, D.: Three different modes of avatars as virtual lecturers in elearning interfaces: a comparative usability study. Open Virtual Reality J. 2, 8–17 (2010)

    Article  Google Scholar 

  12. Lusk, M.M., Atkinson, R.K.: Varying a pedagogical agent’s degree of embodiment under two visual search conditions. Appl. Cogn. Psychol. 21, 747–764 (2007)

    Article  Google Scholar 

  13. Dehn, D.M., Van Mulken, S.: Impact of animated interface agents: a review of empirical research. Int. J. Hum. Comput. Stud. (2000). https://doi.org/10.1006/ijhc.1999.0325

    Article  Google Scholar 

  14. Poggiali, J.: Student responses to an animated character in information literacy instruction. Library Hi Tech 36 (2018). https://doi.org/10.1108/lht-12-2016-0149

  15. Mayer, R.E., Estrella, G.: Benefits of emotional design in multimedia instruction. Learn. Instr. 33, 12–18 (2014)

    Article  Google Scholar 

  16. Mayer, R.E., DaPra, C.S.: An embodiment effect in computer-based learning with animated pedagogical agents. J. Exp. Psychol. Appl. 18(3), 239–252 (2012)

    Article  Google Scholar 

  17. Gulz, A., Haake, M.: Social and visual style in virtual pedagogical agents. In: Proceedings of the Workshop on Adapting the Interaction Style to Affective Factors, 10th International Conference on User Modelling (2005)

    Google Scholar 

  18. Domagk, S.: Do pedagogical agents facilitate learner motivation and learning outcomes? The role of the appeal of agent’s appearance and voice. J. Media Psychol. 22(2), 84–97 (2010)

    Article  Google Scholar 

  19. Mayer, R.E.: Principles based on social cues in multimedia learning: personalization, voice, image, and embodiment principles. In: Mayer, R.E. (ed.) The Cambridge Handbook of Multimedia Learning, 2nd edn, pp. 345–368. Cambridge University Press, New York (2014)

    Chapter  Google Scholar 

  20. Rosenberg-Kima, R.B., Baylor, A.L., Plant, E.A., Doerr, C.E.: Interface agents as social models for female students: the effects of agent visual presence and appearance on female students’ attitudes and beliefs. Comput. Hum. Behav. 24, 2741–2756 (2008)

    Article  Google Scholar 

  21. Baylor, A.L., Kim, S.: Designing nonverbal communication for pedagogical agents: when less is more. Comput. Hum. Behav. 25(2), 450–457 (2009)

    Article  Google Scholar 

  22. Wang, N., Johnson, W.L., Mayer, R.E., Rizzo, P., Shaw, E., Collins, H.: The politeness effect: pedagogical agents and learning outcomes. Int. J. Hum. Comput. Stud. 66, 98–112 (2008)

    Article  Google Scholar 

  23. Larsson, P.: Discerning emotion through movement – a study of body language in portraying emotion in animation, pp. 6–7, May 2014. http://www.diva-portal.org/smash/get/diva2:723103/FULLTEXT01.pdf

  24. Anasingaraju, S., Popescu, V., Adamo, N., Wu, M.L.: Digital learning activities delivered by eloquent instructor avatars: scaling with problem instance. In: Proceedings of SIGGRAPH Asia 2016 – Education Symposium. ACM Digital Library (2016)

    Google Scholar 

  25. Karg, M., Samadani, A.A., Gorbet, R., Kühnlenz, K., Hoey, J., Kulić, D.: Body movements for affective expression: a survey of automatic recognition and generation. IEEE Trans. Affect. Comput. 4(4), 341–359 (2013)

    Article  Google Scholar 

  26. Sawada, M., Suda, K., Ishii, M.: Expression of emotions in dance: relation between arm movement characteristics and emotion. Percept. Mot. Skills 97, 697–708 (2003)

    Article  Google Scholar 

  27. Atkinson, A.P., Dittrich, W.H., Gemmell, A.J., Young, A.W.: Emotion perception from dynamic and static body expressions in point-light and full-light displays. Perception 33(6), 717–746 (2004). https://doi.org/10.1068/p5096

    Article  Google Scholar 

  28. Ennis, C., Hoyet, L., Egges, A., McDonnell, R.: Emotion capture: emotionally expressive characters for games. In: Proceedings of Motion on Games (2013). https://doi.org/10.1145/2522628.2522633

  29. André, E., Klesen, M., Gebhard, P., Allen, S., Rist, T.: Exploiting models of personality and emotions to control the behavior of animated interactive agents. In: Workshop on “Achieving Human-Like Behavior in Interactive Animated Agents” in Conjunction with the Fourth International Conference on Autonomous Agents, pp. 3–7 (2000)

    Google Scholar 

  30. Allbeck, J., Badler, N.: Toward representing agent behaviors modified by personality and emotion. In: Embodied Conversational Agents at AAMAS, vol. 2, pp. 15–19 (2002). https://doi.org/10.1.1.19.2054

    Google Scholar 

  31. Mehrabian, A.: Analysis of the big-five personality factors in terms of the PAD temperament model. Aust. J. Psychol. (1996). https://doi.org/10.1080/00049539608259510

    Article  Google Scholar 

  32. Neff, M., Wang, Y., Abbott, R., Walker, M.: Evaluating the effect of gesture and language on personality perception in conversational agents. In: Allbeck, J., Badler, N., Bickmore, T., Pelachaud, C., Safonova, A. (eds.) IVA 2010. LNCS (LNAI), vol. 6356, pp. 222–235. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15892-6_24

    Chapter  Google Scholar 

  33. Ball, G., Breese, J.: Relating personality and behavior: posture and gestures, pp. 196–203 (2006). https://doi.org/10.1007/10720296_14

  34. Roberts, S.: Character Animation in 3D. Focal Press, Oxford (2004)

    Book  Google Scholar 

  35. Cui, J., Adamo, N., Popescu, V.: Charismatic and eloquent instructor avatars with scriptable gesture. In: Proceedings of SIGGRAPH 2014 - Talks, Vancouver, August 2014 (2014)

    Google Scholar 

  36. Kim, Y.: Pedagogical agents as learning companions: the effects of agent affect and gender on learning, interest, self-efficacy, and agent persona. Ph.D. thesis, FSU (2003). https://diginole.lib.fsu.edu/islandora/object/fsu:181467/datastream/PDF/view

Download references

Acknowledgements

The work reported in the paper is supported in part by NSF – Cyberlearning Collaborative Research: Multimodal Affective Pedagogical Agents for Different Types of Learners, Award Number: 1821894. We thank Purdue Statistical Consulting Services for their help with the statistical analyses.

Author information

Authors and Affiliations

  1. Purdue University, West Lafayette, IN, 47906, USA

    Justin Cheng, Wenbin Zhou, Xingyu Lei, Nicoletta Adamo & Bedrich Benes

Authors
  1. Justin Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wenbin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingyu Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nicoletta Adamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bedrich Benes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicoletta Adamo .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cheng, J., Zhou, W., Lei, X., Adamo, N., Benes, B. (2020). The Effects of Body Gestures and Gender on Viewer’s Perception of Animated Pedagogical Agent’s Emotions. In: Kurosu, M. (eds) Human-Computer Interaction. Multimodal and Natural Interaction. HCII 2020. Lecture Notes in Computer Science(), vol 12182. Springer, Cham. https://doi.org/10.1007/978-3-030-49062-1_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-49062-1_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-49061-4

  • Online ISBN: 978-3-030-49062-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation