PuzzleAide: Comparing Audio and Embodied Assistants for MR Puzzle-Solving
Authors: 
Shirin Hajahmadi, Fariba Mostajeran, Kevin Heuer, Anton Lux, Gil Otis Mends-Cole, Pasquale Cascarano, 
Frank Steinicke, Gustavo MarfiaAuthors Info & Claims
Shirin Hajahmadi, Fariba Mostajeran, Kevin Heuer, Anton Lux, Gil Otis Mends-Cole, Pasquale Cascarano, Frank Steinicke, Gustavo MarfiaAuthors Info & ClaimsArticle No.: 50, Pages 1 - 3
Abstract
Conversational Virtual Agents (CVA) have great potential to assist users in their task performance. Whether these agents need to be embodied or not was the main research question of the present pilot study. To answer this question, we designed and developed a Mixed Reality (MR) application for solving a physical puzzle while interacting with a CVA. Eleven participants took part in our between-subject pilot study and interacted with two different representations of a CVA (i.e., voice-only and embodied agent). In this short paper, we descriptively report on the participants’ problem-solving time, the number of assistance requests they made to the CVAs, and the social presence they perceived while doing so.
References
[1]
Mahoro Anabuki, Hiroyuki Kakuta, Hiroyuki Yamamoto, and Hideyuki Tamura. 2000. Welbo: An embodied conversational agent living in mixed reality space. In CHI’00 extended abstracts on Human factors in computing systems. 10–11.
[2]
Amaël Arguel, Lori Lockyer, Kevin Chai, Mariya Pachman, and Ottmar V Lipp. 2019. Puzzle-solving activity as an indicator of epistemic confusion. Frontiers in psychology 10 (2019), 163.
[3]
Jürgen Bohn. 2004. The smart jigsaw puzzle assistant: Using RFID technology for building augmented real-world games. In Workshop on Gaming Applications in Pervasive Computing Environments at Pervasive, Vol. 2004. Citeseer.
[4]
Chih Yueh Chou, Chang Ching Yang, and Zhi Hong Chen. 2013. Applying augmented reality to assisting children in solving Tangram puzzle. In Proceedings of the 21st International Conference on Computers in Education. 139–144.
[5]
Yuze Dan, Zhenjiang Shen, Yiyun Zhu, and Ling Huang. 2021. Using mixed reality (MR) to improve on-site design experience in community planning. Applied Sciences 11, 7 (2021), 3071.
[6]
Shirin Hajahmadi, Luca Clementi, María Dolores Jiménez López, and Gustavo Marfia. 2024. ARELE-bot: Inclusive Learning of Spanish as a Foreign Language Through a Mobile App Integrating Augmented Reality and ChatGPT. In 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 335–340.
[7]
Chad Harms and Frank Biocca. 2004. Internal consistency and reliability of the networked minds measure of social presence. In Seventh annual international workshop: Presence, Vol. 2004. Universidad Politecnica de Valencia Valencia, Spain.
[8]
Thomas Holz, Abraham G Campbell, Gregory MP O’Hare, John W Stafford, Alan Martin, and Mauro Dragone. 2011. Mira—mixed reality agents. International journal of human-computer studies 69, 4 (2011), 251–268.
[9]
Jon-Chao Hong, Ming-Yueh Hwang, Ker-Ping Tam, Yi-Hsuan Lai, and Li-Chun Liu. 2012. Effects of cognitive style on digital jigsaw puzzle performance: A GridWare analysis. Computers in Human Behavior 28, 3 (2012), 920–928.
[10]
Owen WS Huang, Hercy NH Cheng, and Tak-Wai Chan. 2007. Number jigsaw puzzle: A mathematical puzzle game for facilitating players’ problem-solving strategies. In 2007 First IEEE International Workshop on Digital Game and Intelligent Toy Enhanced Learning (DIGITEL’07). IEEE, 130–134.
[11]
Lucie Kruse, Julia Hertel, Fariba Mostajeran, Susanne Schmidt, and Frank Steinicke. 2023. Would You Go to a Virtual Doctor? A Systematic Literature Review on User Preferences for Embodied Virtual Agents in Healthcare. In 2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). IEEE, 672–682.
[12]
Lucie Kruse, Fariba Mostajeran, and Frank Steinicke. 2023. High Levels of Visibility of Virtual Agents Increase the Social Presence of Users. In 2023 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 843–844.
[13]
Lucie Kruse, Fariba Mostajeran, and Frank Steinicke. 2023. The Influence of Virtual Agent Visibility in Virtual Reality Cognitive Training. In Proceedings of the 2023 ACM Symposium on Spatial User Interaction. 1–9.
[14]
Sheetal Kusal, Shruti Patil, Jyoti Choudrie, Ketan Kotecha, Sashikala Mishra, and Ajith Abraham. 2022. AI-based conversational agents: A scoping review from technologies to future directions. IEEE Access 10 (2022), 92337–92356.
[15]
Liliana Laranjo, Adam G Dunn, Huong Ly Tong, Ahmet Baki Kocaballi, Jessica Chen, Rabia Bashir, Didi Surian, Blanca Gallego, Farah Magrabi, Annie YS Lau, 2018. Conversational agents in healthcare: a systematic review. Journal of the American Medical Informatics Association 25, 9 (2018), 1248–1258.
[16]
João Paulo Lima, João Marcelo Teixeira, and Veronica Teichrieb. 2014. AR jigsaw puzzle with RGB-D based detection of texture-less pieces. In 2014 IEEE Virtual Reality (VR). IEEE, 177–178.
[17]
Chien-Heng Lin and Chien-Min Chen. 2016. Developing spatial visualization and mental rotation with a digital puzzle game at primary school level. Computers in Human Behavior 57 (2016), 23–30.
[18]
Pattie Maes, Robert H Guttman, and Alexandros G Moukas. 1999. Agents that buy and sell. Commun. ACM 42, 3 (1999), 81–ff.
[19]
Mark Roman Miller, Hanseul Jun, Fernanda Herrera, Jacob Yu Villa, Greg Welch, and Jeremy N Bailenson. 2019. Social interaction in augmented reality. PloS one 14, 5 (2019), e0216290.
[20]
Fariba Mostajeran, Nadia Burke, Nazife Ertugrul, Kilian Hildebrandt, Joshua Matov, Noémie Tapie, Wilhelm Gottlieb Zittel, Pia Reisewitz, and Frank Steinicke. 2022. Anthropomorphism of virtual agents and human cognitive performance in augmented reality. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 329–332.
[21]
Fariba Mostajeran, Pia Reisewitz, and Frank Steinicke. 2022. Social facilitation and inhibition in augmented reality: performing motor and cognitive tasks in the presence of a virtual agent. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW). IEEE, 323–328.
[22]
Fariba Mostajeran, Frank Steinicke, Oscar Javier Ariza Nunez, Dimitrios Gatsios, and Dimitrios Fotiadis. 2020. Augmented reality for older adults: exploring acceptability of virtual coaches for home-based balance training in an aging population. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. 1–12.
[23]
Alexandra A Muller and Marion Perlmutter. 1985. Preschool children’s problem-solving interactions at computers and jigsaw puzzles. Journal of Applied Developmental Psychology 6, 2-3 (1985), 173–186.
[24]
Anderson Nishihara and Jun Okamoto. 2015. Object recognition in assembly assisted by augmented reality system. In 2015 SAI Intelligent Systems Conference (IntelliSys). IEEE, 400–407.
[25]
Somaiieh Rokhsaritalemi, Abolghasem Sadeghi-Niaraki, and Soo-Mi Choi. 2020. A review on mixed reality: Current trends, challenges and prospects. Applied Sciences 10, 2 (2020), 636.
[26]
João MXN Teixeira, Pedro JL Silva, Júlia DT de Souza, Filipe F Monteiro, and Veronica Teichrieb. 2020. JigsAR: A Mixed Reality System for Supporting the Assembly of Jigsaw Puzzles. In Design, User Experience, and Usability. Design for Contemporary Interactive Environments: 9th International Conference, DUXU 2020, Held as Part of the 22nd HCI International Conference, HCII 2020, Copenhagen, Denmark, July 19–24, 2020, Proceedings, Part II 22. Springer, 572–585.
[27]
Rainer Winkler, Sebastian Hobert, Antti Salovaara, Matthias Söllner, and Jan Marco Leimeister. 2020. Sara, the lecturer: Improving learning in online education with a scaffolding-based conversational agent. In Proceedings of the 2020 CHI conference on human factors in computing systems. 1–14.
Index Terms
- PuzzleAide: Comparing Audio and Embodied Assistants for MR Puzzle-Solving
Recommendations
Enabling Embodied Conversational Agents to Respond to Nonverbal Behavior of the Communication Partner
Human-Computer Interaction. User Experience and BehaviorAbstractHumans communicate on three levels: words, paralanguage, and nonverbal. While conversational agents focus mainly on the interpretation of words that are being spoken, recently the focus has also shifted to how we say those words with our tone, ...
Let Your World Open: CAVE-based Visualization Methods of Public Virtual Reality towards a Shareable VR Experience
AH2019: Proceedings of the 10th Augmented Human International Conference 2019Virtual reality (VR) games are currently becoming part of the public-space entertainment (e.g., VR amusement parks). Therefore, VR games should be attractive for players, as well as for bystanders. Current VR systems are still mostly focused on ...
Modeling situation-dependent nonverbal expressions for a pair of embodied agent in a dialogue based on conversations in TV programs
HCI'13: Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IVMathematical model for controlling nonverbal expressions of a pair of embodied agents designed for presenting various information through their dialogue is discussed. Nonverbal expressions of a human during conversation with others depend on those of ...
Comments
Information & Contributors
Information
Published In
October 2024
396 pages
ISBN:9798400710889
DOI:10.1145/3677386
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 October 2024
Check for updates
Author Tags
Qualifiers
- Abstract
- Research
- Refereed limited
Conference
Acceptance Rates
Overall Acceptance Rate 86 of 279 submissions, 31%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 58Total Downloads
- Downloads (Last 12 months)58
- Downloads (Last 6 weeks)8
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format