skip to main content
10.1145/3534085.3534340acmconferencesArticle/Chapter ViewAbstractPublication PagesmmsysConference Proceedingsconference-collections
research-article

Towards an AI-driven talking avatar in virtual reality for investigative interviews of children

Published: 17 June 2022 Publication History

Abstract

Artificial intelligence (AI) and gaming systems have advanced to the stage where the current models and technologies can be used to address real-world problems. The development of such systems comes with different challenges, e.g., most of them related to system performance, complexity and user testing. Using a virtual reality (VR) environment, we have designed and developed a game-like system aiming to mimic an abused child that can help to assist police and child protection service (CPS) personnel in interview training of maltreated children. Current research in this area points to the poor quality of conducted interviews, and emphasises the need for better training methods. Information obtained in these interviews is the core piece of evidence in the prosecution process. We utilised advanced dialogue models, talking visual avatars, and VR to build a virtual child avatar that can interact with users. We discuss our proposed architecture and the performance of the developed child avatar prototype, and we present the results from the user study conducted with CPS personnel. The user study investigates the users' perceived quality of experience (QoE) and their learning effects. Our study confirms that such a gaming system can increase the knowledge and skills of the users. We also benchmark and discuss the system performance aspects of the child avatar. Our results show that the proposed prototype works well in practice and is well received by the interview experts.

References

[1]
Gunn-Astrid Baugerud, Miriam S Johnson, Helle BG Hansen, Svein Magnussen, and Michael E Lamb. 2020. Forensic interviews with preschool children: An analysis of extended interviews in Norway (2015--2017). Applied Cognitive Psychology 34, 3 (2020), 654--663.
[2]
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165 (2020).
[3]
Nathalie Carrick, Jodi A Quas, and Thomas Lyon. 2010. Maltreated and nonmal-treated children's evaluations of emotional fantasy. Child Abuse & Neglect 34, 2 (2010), 129.
[4]
Ann-Christin Cederborg, Yael Orbach, Kathleen J Sternberg, and Michael E Lamb. 2000. Investigative interviews of child witnesses in Sweden. Child abuse & neglect 24, 10 (2000), 1355--1361.
[5]
Dante Cicchetti and Sheree L Toth. 2005. Child maltreatment. Annu. Rev. Clin. Psychol. 1 (2005), 409--438.
[6]
Kevin Charles Dalli. 2021. Technological Acceptance of an Avatar Based Interview Training Application: The development and technological acceptance study of the AvBIT application.
[7]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[8]
Pierre Gander. 1999. Two myths about immersion in new storytelling media. Lund University.
[9]
Sebastian Hobert. 2019. Say hello to 'coding tutor'! design and evaluation of a chatbot-based learning system supporting students to learn to program. (2019).
[10]
Wijnand A IJsselsteijn, Yvonne AW de Kort, and Karolien Poels. 2013. The game experience questionnaire. Eindhoven: Technische Universiteit Eindhoven 46, 1 (2013).
[11]
ITU-T Recommendation P.809. 2018. Subjective Evaluation Methods for Gaming Quality. International Telecommunication Union, Geneva.
[12]
David Johansson. 2015. Design and evaluation of an avatar-mediated system for child interview training.
[13]
Miriam Johnson, Svein Magnussen, Christian Thoresen, Kyrre Lønnum, Lisa Victoria Burrell, and Annika Melinder. 2015. Best practice recommendations still fail to result in action: A national 10-year follow-up study of investigative interviews in CSA cases. Applied Cognitive Psychology 29, 5 (2015), 661--668.
[14]
Yael Karni-Visel, Irit Hershkowitz, Michael E Lamb, and Uri Blasbalg. 2021. Nonverbal emotions while disclosing child abuse: the role of interviewer support. Child maltreatment (2021), 10775595211063497.
[15]
Friedemann Köster, Dennis Guse, Marcel Wältermann, and Sebastian Möller. 2015. Comparison between the discrete ACR scale and an extended continuous scale for the quality assessment of transmitted speech. Fortschritte der Akustik, DAGA (2015).
[16]
Michael E Lamb. 2016. Difficulties translating research on forensic interview practices to practitioners: Finding water, leading horses, but can we get them to drink? American psychologist 71, 8 (2016), 710.
[17]
Michael E Lamb, Deirdre A Brown, Irit Hershkowitz, Yael Orbach, and Phillip W Esplin. 2018. Tell me what happened: Questioning children about abuse. John Wiley & Sons.
[18]
Michael E Lamb, Yael Orbach, Irit Hershkowitz, Phillip W Esplin, and Dvora Horowitz. 2007. A structured forensic interview protocol improves the quality and informativeness of investigative interviews with children: A review of research using the NICHD Investigative Interview Protocol. Child abuse & neglect 31, 11--12 (2007), 1201--1231.
[19]
Karl F MacDorman, Robert D Green, Chin-Chang Ho, and Clinton T Koch. 2009. Too real for comfort? Uncanny responses to computer generated faces. Computers in human behavior 25, 3 (2009), 695--710.
[20]
Fernando A Mikic-Fonte, Martín Llamas-Nistal, and Manuel Caeiro-Rodríguez. 2018. Using a Chatterbot as a FAQ Assistant in a Course about Computers Architecture. In 2018 IEEE Frontiers in Education Conference (FIE). IEEE, 1--4.
[21]
Masahiro Mori, Karl F MacDorman, and Norri Kageki. 2012. The uncanny valley [from the field]. IEEE Robotics & Automation Magazine 19, 2 (2012), 98--100.
[22]
Chris Newlin, Linda Cordisco Steele, Andra Chamberlin, Jennifer Anderson, Julie Kenniston, Amy Russell, Heather Stewart, and Viola Vaughan-Eden. 2015. Child forensic interviewing: Best practices. US Department of Justice, Office of Justice Programs, Office of Juvenile ....
[23]
Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499 (2016).
[24]
Andrew Perkis, Christian Timmerer, Sabina Baraković, Jasmina Baraković Husić, Søren Bech, Sebastian Bosse, Jean Botev, Kjell Brunnström, Luis Cruz, Katrien De Moor, et al. 2020. QUALINET white paper on definitions of immersive media experience (IMEx). arXiv preprint arXiv:2007.07032 (2020).
[25]
Francesco Pompedda, Angelo Zappalà, and Pekka Santtila. 2015. Simulations of child sexual abuse interviews using avatars paired with feedback improves interview quality. Psychology, Crime & Law 21, 1 (2015), 28--52.
[26]
Martine B Powell, Belinda Guadagno, and Mairi Benson. 2016. Improving child investigative interviewer performance through computer-based learning activities. Policing and Society 26, 4 (2016), 365--374.
[27]
Alec Radford, Karthik Narasimhan, Tim Salimans, and Ilya Sutskever. 2018. Improving language understanding by generative pre-training. (2018).
[28]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 8 (2019), 9.
[29]
Marie-Laure Ryan. 1999. Immersion vs. interactivity: Virtual reality and literary theory. SubStance 28, 2 (1999), 110--137.
[30]
Saeed Shafiee Sabet, Steven Schmidt, Saman Zadtootaghaj, Carsten Griwodz, and Sebastian Möller. 2020. Delay sensitivity classification of cloud gaming content. In Proceedings of the 12th ACM International Workshop on Immersive Mixed and Virtual Environment Systems. 25--30.
[31]
Saeed Shafiee Sabet, Steven Schmidt, Saman Zadtootaghaj, Babak Naderi, Carsten Griwodz, and Sebastian Möller. 2020. A latency compensation technique based on game characteristics to mitigate the influence of delay on cloud gaming quality of experience. In Proceedings of the 11th ACM Multimedia Systems Conference. 15--25.
[32]
Pegah Salehi, Syed Zohaib Hassan, Saeed Shafiee Sabet, Gunn Astrid Baugerud, Miriam Sinkerud Johnson, Michael A. Riegler, and Pål Halvorsen. 2022. Is More Realistic Better? A Comparison of Game Engine and GAN-based Avatars for Investigative Interviews of Children. Workshop on Intelligent Cross-Data Analysis and Retrieval (2022).
[33]
Karen J Saywitz, Thomas D Lyon, and Gail S Goodman. 2017. 19 When Interviewing Children: A Review and Update. The APSAC handbook on child maltreatment (2017), 310.
[34]
Steven Schmidt. 2021. Assessing the Quality of Experience of Cloud Gaming Services. P.h.D. thesis, Technische Universität Berlin.
[35]
Jack P Shonkoff. 2016. Capitalizing on advances in science to reduce the health consequences of early childhood adversity. JAMA pediatrics 170, 10 (2016), 1003--1007.
[36]
Sharob Sinha, Shyanka Basak, Yajushi Dey, and Anupam Mondal. 2020. An educational Chatbot for answering queries. In Emerging Technology in Modelling and Graphics. Springer, 55--60.
[37]
Kathleen J Sternberg, Michael E Lamb, Yael Orbach, Phillip W Esplin, and Susanne Mitchell. 2001. Use of a structured investigative protocol enhances young children's responses to free-recall prompts in the course of forensic interviews. Journal of applied psychology 86, 5 (2001), 997.
[38]
Sheree L Toth and Dante Cicchetti. 2013. A developmental psychopathology perspective on child maltreatment. Child maltreatment 18, 3 (2013), 135--139.
[39]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems. 5998--6008.
[40]
Oriol Vinyals and Quoc Le. 2015. A neural conversational model. arXiv preprint arXiv:1506.05869 (2015).
[41]
WHO. 2020. Child maltreatment. https://www.who.int/news-room/fact-sheets/detail/child-maltreatment
[42]
Mai Xu, Chen Li, Zhenzhong Chen, Zulin Wang, and Zhenyu Guan. 2018. Assessing visual quality of omnidirectional videos. IEEE transactions on circuits and systems for video technology 29, 12 (2018), 3516--3530.
[43]
Hao Zhou, Minlie Huang, Tianyang Zhang, Xiaoyan Zhu, and Bing Liu. 2018. Emotional chatting machine: Emotional conversation generation with internal and external memory. In Thirty-Second AAAI Conference on Artificial Intelligence.

Cited By

View all
  • (2025)Virtual Reality Training as Enhanced Experiential LearningJournal of Technology in Behavioral Science10.1007/s41347-024-00477-9Online publication date: 5-Feb-2025
  • (2025) Using an AI ‐Driven Child Chatbot Avatar as a Training Tool for Information Gathering Skills of Dental and Medical Professionals: A Pilot Study Applied Cognitive Psychology10.1002/acp.7002239:1Online publication date: 7-Jan-2025
  • (2024)Enhancing Inclusivity in Interviewing: Harnessing Intelligent Digital Avatars for Bias MitigationAdvances in Digital Transformation - Rise of Ultra-Smart Fully Automated Cyberspace10.5772/intechopen.1004393Online publication date: 6-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
GameSys '22: Proceedings of the 2nd Workshop on Games Systems
June 2022
34 pages
ISBN:9781450393812
DOI:10.1145/3534085
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 June 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. AI
  2. avatar
  3. child protection services (CPS)
  4. dialogue model
  5. generative adversarial networks (GANs)
  6. quality of experience (QoE)
  7. virtual reality (VR)

Qualifiers

  • Research-article

Funding Sources

  • Research Council of Norway

Conference

MMSys '22
Sponsor:

Acceptance Rates

Overall Acceptance Rate 4 of 7 submissions, 57%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)266
  • Downloads (Last 6 weeks)28
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Virtual Reality Training as Enhanced Experiential LearningJournal of Technology in Behavioral Science10.1007/s41347-024-00477-9Online publication date: 5-Feb-2025
  • (2025) Using an AI ‐Driven Child Chatbot Avatar as a Training Tool for Information Gathering Skills of Dental and Medical Professionals: A Pilot Study Applied Cognitive Psychology10.1002/acp.7002239:1Online publication date: 7-Jan-2025
  • (2024)Enhancing Inclusivity in Interviewing: Harnessing Intelligent Digital Avatars for Bias MitigationAdvances in Digital Transformation - Rise of Ultra-Smart Fully Automated Cyberspace10.5772/intechopen.1004393Online publication date: 6-Mar-2024
  • (2024)Using an AI-based avatar for interviewer training at Children’s Advocacy Centers: Proof of ConceptChild Maltreatment10.1177/10775595241263017Online publication date: 18-Jun-2024
  • (2024)An Empirical Study on Social Anxiety in a Virtual Environment through Mediating Variables and Multiple Sensor DataProceedings of the ACM on Human-Computer Interaction10.1145/36869778:CSCW2(1-24)Online publication date: 8-Nov-2024
  • (2024)Beyond Reality: The Pivotal Role of Generative AI in the MetaverseIEEE Internet of Things Magazine10.1109/IOTM.001.23001747:4(126-135)Online publication date: Jul-2024
  • (2024)A Theoretical and Empirical Analysis of 2D and 3D Virtual Environments in Training for Child Interview SkillsIEEE Access10.1109/ACCESS.2024.344229712(131842-131864)Online publication date: 2024
  • (2024)Exploring AI Interaction Modalities in Virtual Environments and Its Impact on Users’ AttitudesThe Impact of Artificial Intelligence on Societies10.1007/978-3-031-70355-3_4(41-55)Online publication date: 19-Dec-2024
  • (2023)ARTIFICIAL INTELLIGENCE AVATAR FOR CONVERSATIONAL AGENT2023 Research, Invention, and Innovation Congress: Innovative Electricals and Electronics (RI2C)10.1109/RI2C60382.2023.10355967(33-39)Online publication date: 24-Aug-2023
  • (2023)Free-form Conversation with Human and Symbolic Avatars in Mixed Reality2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00090(751-760)Online publication date: 16-Oct-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media