Welcome to the InfoSci Platform
Multimodal Data Integration and User Interaction for Avatar Simulation in Augmented Reality

Multimodal Data Integration and User Interaction for Avatar Simulation in Augmented Reality

Anchen Sun, Yudong Tao, Mei-Ling Shyu, Angela Blizzard, William Andrew Rothenberg, Dainelys Garcia, Jason F. Jent
Copyright: © 2022 |Volume: 13 |Issue: 1 |Pages: 19
ISSN: 1947-8534|EISSN: 1947-8542|EISBN13: 9781683180852|DOI: 10.4018/IJMDEM.304391
Cite Article Cite Article

MLA

Sun, Anchen, et al. "Multimodal Data Integration and User Interaction for Avatar Simulation in Augmented Reality." IJMDEM vol.13, no.1 2022: pp.1-19. http://doi.org/10.4018/IJMDEM.304391

APA

Sun, A., Tao, Y., Shyu, M., Blizzard, A., Rothenberg, W. A., Garcia, D., & Jent, J. F. (2022). Multimodal Data Integration and User Interaction for Avatar Simulation in Augmented Reality. International Journal of Multimedia Data Engineering and Management (IJMDEM), 13(1), 1-19. http://doi.org/10.4018/IJMDEM.304391

Chicago

Sun, Anchen, et al. "Multimodal Data Integration and User Interaction for Avatar Simulation in Augmented Reality," International Journal of Multimedia Data Engineering and Management (IJMDEM) 13, no.1: 1-19. http://doi.org/10.4018/IJMDEM.304391

Export Reference

Mendeley
Favorite Full-Issue Download

Abstract

Augmented Reality (AR) allows users to interact with the virtual world in the real-world environment. This paper proposes a child avatar simulation framework using multimodal data integration and user interaction in the AR environment. This framework generates the child avatar that can interact with the user and respond to his/her behaviors and consists of three subsystems: (1) the avatar interaction system scrutinizes user behaviors based on the user data, (2) the avatar action control system generates naturalistic avatar activities (actions and voices) according to the avatar internal status, and (3) the avatar display system renders the avatar through the AR interface. In addition, a child tantrum management training application has been built based on the proposed framework. And a light machine learning model has been integrated to enable efficient and effective speech emotion recognition. A sufficiently realistic child tantrum management training based on the evaluation of clinical child psychologists is enabled, which helps users get familiar with child tantrum management.

Request Access

You do not own this content. Please login to recommend this title to your institution's librarian or purchase it from the IGI Global bookstore.