Skip to main content
SpringerLink
Log in

Virtual agents as supporting media for scientific presentations

  • Original Paper
  • Published:
Journal on Multimodal User Interfaces Aims and scope Submit manuscript

Abstract

The quality of scientific oral presentations is often poor, owing to a number of factors, including public speaking anxiety. We present DynamicDuo, a system that uses an automated, life-sized, animated agent to help inexperienced scientists deliver their presentations in front of an audience. The design of the system was informed by an analysis of TED talks given by pairs of human presenters to identify the most common dual-presentation formats and transition behaviors used. We explore the usability and acceptability of DynamicDuo in both controlled laboratory-based studies and real-world environments, and its ability to decrease public speaking anxiety and improve presentation quality. In a within-subjects study (N = 12) comparing co-presenting with DynamicDuo against solo-presenting with conventional presentation software, we demonstrated that our system led to significant improvements in public speaking anxiety and speaking confidence for non-native English speakers. Judges who viewed videotapes of these presentations rated those with DynamicDuo significantly higher on speech quality and overall presentation quality for all presenters. We also explore the affordances of the virtual co-presenter through empirical evaluation of novel roles the agent can play in scientific presentations and novel ways it can interact with the speaker in front of the audience.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Goodman A (2006) Why bad presentations happen to good causes. Andy Goodman & Cause Communication

  2. Bishop J, Bauer K, Becker E (1998) A survey of counseling needs of male and female college students. J College Stud Dev 39:205–210

    Google Scholar 

  3. Weingart P, Joubert M (2019) The conflation of motives of science communication: Causes, consequences, remedies. J Sci Commun 18(3):1–13

    Article  Google Scholar 

  4. National Academies of Sciences Engineering, and Medicine (2017) Communicating science effectively: a research agenda. The National Academies Press, Washington, DC

    Google Scholar 

  5. Niemann P, Bittner L, Schrögel P, Hauser C (2020) Science slams as edutainment: a reception study. Media Commun 8(1):177–190

    Article  Google Scholar 

  6. Jackson J, Latané B (1981) All alone in front of all those people: stage fright as a function of number and type of co-performers and audience. J Pers Soc Psychol 40(1):73–81

    Article  Google Scholar 

  7. Cassell J, Sullivan J, Prevost S, Churchill E (eds) (2000) Embodied conversational agents. The MIT Press, Cambridge, MA

    Google Scholar 

  8. Edge D, Savage J, Yatani K (2013) HyperSlides: dynamic presentation prototyping. Paper presented at CHI’13

  9. Trinh H, Yatani K, Edge D (2014) PitchPerfect: integrated rehearsal environment for structured presentation preparation. Paper presented at the CHI’14

  10. Saket B, Yang S, Tan H, Yatani K, Edge D (2014) TalkZones: section-based time support for presentations. Paper presented at the MobileHCI 2014 conference on human computer interaction with mobile devices and services

  11. McVeigh-Schultz J, Kolesnichenko A, Isbister K (2019) Shaping pro-social interaction in VR: an emerging design framework. Paper presented at CHI’19

  12. André E, Rist T, Müller J (1998) WebPersona: a lifelike presentation agent for the World-Wide Web. Knowl Based Syst 11(1):25–36

    Article  Google Scholar 

  13. Fourati N, Richard A, Caillou S, Sabouret N, Martin J-C, Chanoni E, Clavel C (2016) Facial expressions of appraisals displayed by a virtual storyteller for children. Paper presented at Intelligent Virtual Agents

  14. Noma T, Zhao L, Badler N (2000) Design of a virtual human presenter. IEEE Comput Graphics Appl 20(4):79–85

    Article  Google Scholar 

  15. Nijholt A, van Welbergen H, Zwiers J (2005) Introducing an embodied virtual presenter agent in a virtual meeting room. Paper presented at the IASTED International Conference on Artificial Intelligence and Applications

  16. Kang N, Brinkman W, van Riemsdijk M, Neerincx M (2013) An expressive virtual audience with flexible behavioral styles. IEEE Trans Affect Comput 4(4):326–340

    Article  Google Scholar 

  17. Pertaub D, Slater M, Barker C (2002) An experiment on public speaking anxiety in response to three different types of virtual audience. Presence Teleoper Virt Environ 11(1):68–78

    Article  Google Scholar 

  18. Chollet M, Scherer S (2017) Perception of virtual audiences. IEEE Comput Graphics Appl 37(4):50–59. https://doi.org/10.1109/MCG.2017.3271465

    Article  Google Scholar 

  19. Pertaub D-P, Slater M, Barker C (2002) An experiment on public speaking anxiety in response to three different types of virtual audience. Pres Teleoper Virt Environ 11(1):68–78

    Article  Google Scholar 

  20. Barreda-Ángeles M, Aleix-Guillaume S, Pereda-Baños A (2019) Users’ psychophysiological, vocal, and self-reported responses to the apparent attitude of a virtual audience in stereoscopic 360°-video. Virt Real 24:289–302

    Article  Google Scholar 

  21. Trinh H, Asadi R, Edge D, Bickmore T (2017) RoboCOP: a robotic coach for oral presentations. Paper presented at the ACM international joint conference on pervasive and ubiquitous computing (Ubicomp)

  22. Hoque M, Courgeon M, Mutlu B, Martin J-C, Picard R (2013) MACH: my automated conversation coach. Paper presented at the 15th international conference on ubiquitous computing (Ubicomp)

  23. Anderson K, André E, Baur T, Bernardini S, Chollet M, Chryssafidou E, Damian I, Ennis C, Egges A, Gebhard P, Jones H, Ochs M, Pelachaud C, Porayska-Pomsta K, Rizzo P, Sabouret N (2013) The TARDIS framework: intelligent virtual agents for social coaching in job interviews. In: Cham, 2013. Advances in computer entertainment. Springer International Publishing, pp 476–491

  24. Duncan S (1974) On the structure of speaker-auditor interaction during speaking turns. Lang Soc 3:161–180

    Article  Google Scholar 

  25. Goodwin C (1981) Achieving mutual orientation at turn beginning. Conversational organization: interaction between speakers and hearers. Academic Press, New York, pp 55–89

    Google Scholar 

  26. Kendon A (1967) Some functions of gaze direction in social interaction. Acta Physiol (Oxf) 26:22–63

    Google Scholar 

  27. Sacks H, Schegloff EA, Jefferson G (1974) A simplest systematics for the organization of turn-taking for conversation. Language 50:696–735

    Article  Google Scholar 

  28. Duncan S (1972) Some signals and rules for taking speaking turns in conversations. J Pers Soc Psychol 23:283–292

    Article  Google Scholar 

  29. Raux A, Eskenazi M (2012) Optimizing the turn-taking behavior of task-oriented spoken dialog systems. ACM Trans Speech Lang Process 9(1):1–23

    Article  Google Scholar 

  30. Ward N, Fuentes O, Vega A (2010) Dialog prediction for a general model of turn-taking. Paper presented at Interspeech

  31. Koiso H, Horiuchi Y, Tutiya S, Ichikawa A, Den Y (1998) An analysis of turn-taking and backchannels based on prosodic and syntactic features in Japanese Map Task dialogs. Lang Speech 41:295–321

    Article  Google Scholar 

  32. Gravano A, Hirschberg J (2011) Turn-taking cues in task-oriented dialogue. Comput Speech Lang 25(3):601–634

    Article  Google Scholar 

  33. Hjalmarsson A (2011) The additive effect of turn-taking cues in human and synthetic voice. Speech Commun 53(1):23–35

    Article  Google Scholar 

  34. Thorisson KR (1997) Gandalf: an embodied humanoid capable of real-time multimodal dialogue with people. Paper presented at autonomous agents ‘97

  35. Cassell J, Bickmore T, Billinghurst M, Campbell L, Chang K, Vilhjalmsson H, Yan H Embodiment in conversational interfaces: rea. Paper presented at CHI’99

  36. Huang L, Morency L, Gratch J (2011) Virtual Rapport 2.0. Paper presented at intelligent virtual agents (IVA)

  37. Jonsdottir G, Thorisson K, Nivel E (2008) Learning smooth, human-like turntaking in realtime dialogue. Paper presented at intelligent virtual agents (IVA)

  38. Chao C, Thomaz A (2016) Timed Petri nets for fluent turn-taking over multimodal interactoin resources in human-robot collaboration. Int J Robot Res 35(11):1330–1353

    Article  Google Scholar 

  39. Jegou M, Lefebvre L, Chevaillier P (2015) A continuous model for the management of turn-taking in user-agent spoken interactions based on the variations of prosodic signals. Paper presented at intelligent virtual agents (IVA)

  40. Trinh H, Ring L, Bickmore T (2015) DynamicDuo: co-presenting with virtual agents. Paper presented at CHI’15

  41. Murali P, Ring L, Trinh H, Asadi R, Bickmore T (2018) Speaker hand-offs in collaborative human-agent oral presentations. Paper presented at the intelligent virtual agents (IVA)

  42. Cassell J, Vilhjálmsson H, Bickmore T (2001) BEAT: the behavior expression animation toolkit. Paper presented at SIGGRAPH ‘01

  43. Bickmore T, Trinh H, Hoppmann M, Asadi R (2016) Virtual agents in the classroom: experience fielding a co-presenter agent in university courses. Paper presented at intelligent virtual agents (IVA)

  44. Spielberger C (1989) State-trait anxiety inventory: bibliography. Consulting Psychologists Press, Palo Alto, CA

    Google Scholar 

  45. Paul G (1966) Insight and desensitization in psychotherapy: an experiment in anxiety reduction. Stanford University Press, Stanford, CA

    Google Scholar 

  46. Goffman E (1981) Forms of talk. University of Pennsylvania Press, Philadelphia, pp 124–159

    Google Scholar 

  47. Kimani E, Shamekhi A, Parmar D, Bickmore T (2019) Stagecraft for scientists: exploring novel interaction formats for virtual co-presenter agents. Paper presented at intelligent virtual agents (IVA)

  48. Holtzblatt K, Wendell J, Wood S (2004) Rapid contextual design: a how-to guide to key techniques for user-centered design. Elsevier, New York

    Google Scholar 

  49. Bakhtin M (1986) The Problem of Speech Genres. In: Holquist M, Emerson C (eds) Speech genres and other essays. University of Texas Press, Austin, pp 60–102

    Google Scholar 

  50. Trinh H, Bickmore T, Edge D, Ring L (2016) Thinking outside the box: co-planning scientific presentations with virtual agents. Paper presented at intelligent virtual agents (IVA)

  51. Asadi R, Fell H, Bickmore T, Trinh H (2016) Real-time presentation tracking using semantic keyword spotting. Paper presented at Interspeech

  52. Asadi R, Trinh A, Fell H, Bickmore T (2018) Quester: a speech-based question-answering support system for oral presentations. Paper presented at the international conference on intelligent user interfaces (IUI)

  53. Asadi R, Trinh H, Bickmore T (2017) IntelliPrompter: speech-based dynamic note display interface for oral presentations. Paper presented at the international conference on multimodal interaction (ICMI)

  54. Kimani E, Trinh H, Pedrelli P, Bickmore T (2019) You’ll be great: virtual agent-based cognitive restructuring to reduce public speaking anxiety. Paper presented at the 8th international conference on affective computing & intelligent interaction (ACII)

  55. Kimani E (2019) A sensor-based framework for real-time detection and alleviation of public speaking anxiety. Paper presented at the international conference on affective computing & intelligent interaction (ACII) doctoral consortium

Download references

Acknowledgements

This material is based upon work supported by the National Science Foundation under Grant No. IIS-1514490. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.

Author information

Authors and Affiliations

  1. Khoury College of Computer Sciences, Northeastern University, Boston, MA, USA

    Timothy Bickmore, Everlyne Kimani, Ameneh Shamekhi, Prasanth Murali, Dhaval Parmar & Ha Trinh

Authors
  1. Timothy Bickmore
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Everlyne Kimani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ameneh Shamekhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Prasanth Murali
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dhaval Parmar
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Ha Trinh
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Timothy Bickmore.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bickmore, T., Kimani, E., Shamekhi, A. et al. Virtual agents as supporting media for scientific presentations. J Multimodal User Interfaces 15, 131–146 (2021). https://doi.org/10.1007/s12193-020-00350-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12193-020-00350-y

Keywords

Search

Navigation