Shuai Zou
ABSTRACT
Recent developments in robotics and virtual reality (VR) are making embodied agents familiar, and social behaviors of embodied conversational agents are essential to create mindful daily lives with conversational agents. Especially, natural nonverbal behaviors are required, such as gaze and gesture movement. We propose a novel method to create an agent with human-like gaze as a listener in multi-party conversation, using Hidden Markov Model (HMM) to learn the behavior from real conversation examples. The model can generate gaze reaction according to users' gaze and utterance. We implemented an agent with proposed method, and created VR environment to interact with the agent. The proposed agent reproduced several features of gaze behavior in example conversations. Impression survey result showed that there is at least a group who felt the proposed agent is similar to human and better than conventional methods.
Supplemental Material
- C. Busso, Z. Deng, M. Grimm, U. Neumann, and S. Narayanan. 2007. Rigid Head Motion in Expressive Speech Animation:Analysis and Synthesis. IEEE Transactions on Audio, Speech, and Language Processing 15, 3 (March 2007), 1075--1086. DOI:http://dx.doi.org/10.1109/TASL.2006.885910 Google Scholar
Digital Library
- Carlos Busso, Zhigang Deng, Ulrich Neumann, and Shrikanth Narayanan. 2005. Natural head motion synthesis driven by acoustic prosodic features. Computer Animation and Virtual Worlds 16, 3--4 (2005), 283--290. DOI:http://dx.doi.org/10.1002/cav.80 Google ScholarCross Ref
- J. CASSEL. 1999. The power of a nod and a glance:Envelope vs.emotional feedback in animated conversational agents. Applied Artificial Intelligence 13, 4 (1999), 519--538. http://ci.nii.ac.jp/naid/80011023310/Google ScholarCross Ref
- David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Edward Fast, Alesia Gainer, Kallirroi Georgila, Jonathan Gratch, Arno Hartholt, Margaux Lhommet, Gale Lucas, Stacy C. Marsella, Morbini Fabrizio, Angela Nazarian, Stefan Scherer, Giota Stratou, Apar Suri, David Traum, Rachel Wood, Yuyu Xu, Albert Rizzo, and Louis-Philippe Morency. 2014. SimSensei Kiosk:A Virtual Human Interviewer for Healthcare Decision Support. In Proceedings of the 13th Inter-national Conference on Autonomous Agents and Multiagent Systems (AAMAS 2014). International Foundation for Autonomous Agents and Multiagent Systems, Paris, France, 1061--1068.Google Scholar
- Adam Kendon. 1967. Some functions of gaze-direction in social interaction. Acta Psychologica 26 (1967), 22 -- 63. DOI:http://dx.doi.org/10.1016/0001--6918(67)90005--4Google ScholarCross Ref
- Soh Masuko and Junichi Hoshino. 2007. Head-eye Animation Corresponding to a Conversation for CG Characters. Computer Graphics Forum 26, 3 (2007), 303--312. DOI:http://dx.doi.org/10.1111/j.1467--8659.2007.01052.xGoogle ScholarCross Ref
- Yukiko I. Nakano, Gabe Reinstein, Tom Stocky, and Justine Cassell. 2003. Towards a Model of Face-to-face Grounding. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics - Volume 1 (ACL '03). Association for Computational Linguistics, Stroudsburg, PA, USA, 553--561. DOI:http://dx.doi.org/10.3115/1075096.1075166 Google ScholarDigital Library
- Pronama-Chan. 2014. User Guideline. (2014). http://pronama.azurewebsites.net/pronama/guideline/Google Scholar
- Pupil-Labs. 2014. Pupil Headset. (2014). https://pupil-labs.com/pupil/Google Scholar
- Tomio Watanabe. 2003. Embodied Interaction and Communication Technology Through the Development of E-COSMIC:Embodied Communication System for Mind Connection. Baby Science 2 (2003), 4--12. http://ci.nii.ac.jp/naid/10018152055/Google Scholar
Index Terms
- Conversational Agent Learning Natural Gaze and Motion of Multi-Party Conversation from Example
Recommendations
Gaze awareness in conversational agents: Estimating a user's conversational engagement from eye gaze
Special issue on interaction with smart objects, Special section on eye gaze and conversationIn face-to-face conversations, speakers are continuously checking whether the listener is engaged in the conversation, and they change their conversational strategy if the listener is not fully engaged. With the goal of building a conversational agent ...
Conversational gaze mechanisms for humanlike robots
During conversations, speakers employ a number of verbal and nonverbal mechanisms to establish who participates in the conversation, when, and in what capacity. Gaze cues and mechanisms are particularly instrumental in establishing the participant roles ...
Impact of video editing based on participants' gaze in multiparty conversation
CHI EA '04: CHI '04 Extended Abstracts on Human Factors in Computing SystemsThis paper presents a video cut editing rule based on participants' gaze for establishing video editing rules that can accurately and clearly convey the flow of conversation in multiparty conversations to viewers. Demand is growing to be able to ...
Comments