Yukiko I. Nakano
Skip Abstract Section
Abstract
Gaze is an important nonverbal feedback signal in multiparty face-to-face conversations. It is well known that gaze behaviors differ depending on participation role: speaker, addressee, or side participant. In this study, we focus on dominance as another factor that affects gaze. First, we conducted an empirical study and analyzed its results that showed how gaze behaviors are affected by both dominance and participation roles. Then, using speech and gaze information that was statistically significant for distinguishing the more dominant and less dominant person in an empirical study, we established a regression-based model for estimating conversational dominance. On the basis of the model, we implemented a dominance estimation mechanism that processes online speech and head direction data. Then we applied our findings to human-robot interaction. To design robot gaze behaviors, we analyzed gaze transitions with respect to participation roles and dominance and implemented gaze-transition models as robot gaze behavior generation rules. Finally, we evaluated a humanoid robot that has dominance estimation functionality and determines its gaze based on the gaze models, and we found that dominant participants had a better impression of less dominant robot gaze behaviors. This suggests that a robot using our gaze models was preferred to a robot that was simply looking at the speaker. We have demonstrated the importance of considering dominance in human-robot multiparty interaction.
- Michael Argyle and Mark Cook. 1976. Gaze and Mutual Gaze. Cambridge, Cambridge University Press.Google Scholar
- Michael Argyle and Jean Ann Graham. 1977. The Central Europe experiment—Looking at persons and looking at things. Journal of Environmental Psychology and Nonverbal Behaviour 1 (1977), 6--16.Google ScholarCross Ref
- Rober F. Bales. 1950. Interaction Process Analysis: A Method for the Study of Small Groups. Addison Wesley.Google Scholar
- Nikolaus Bee, Stefan Franke, and Elisabeth Andre. 2009. Relations between facial display, eye gaze and head tilt: Dominance perception variations of virtual agents. In Proceedings of Affective Computing and Intelligent Interaction and Workshops (ACII’09).Google ScholarCross Ref
- Frank J. Bernieri, John S. Gillis, Janet M. Davis, and Jon E. Grahe. 1996. Dyad rapport and the accuracy of its judgment across situations: A lens model analysis. Journal of Personality and Social Psychology 71 (1996) 110--129.Google Scholar
- Dan Bohus and Eric Horvitz. 2009. Dialog in the open world: Platform and applications. In Proceedings of the International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI’09). Google ScholarDigital Library
- Dan Bohus and Eric Horvitz. 2010. Facilitating multiparty dialog with gaze, gesture, and speech. In Proceedings of the International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI’10). Google ScholarDigital Library
- Dan Bohus and Eric Horvitz. 2011. Multiparty turn taking in situated dialog: Study, lessons, and directions. In Proceedings of the SIGDIAL 2011 Conference (SIGDIAL’11). 98--109. Google ScholarDigital Library
- Lei Chen and Mary P. Harper. 2009. Multimodal floor control shift detection. In Proceedings of the 11th International Conference on Multimodal Interfaces and the Sixth Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI’09). Google ScholarDigital Library
- Herbert H. Clark. 1996. Using Language. Cambridge, Cambridge University Press.Google Scholar
- John F. Dovidio and Steve L. Ellyson. 1985. Patterns of visual dominance behavior in humans. In Power, Dominance, and Nonverbal Behavior, S. L. Ellyson and J. F. Dovidio (Ed.). Springer-Verlag, New York. 129--149.Google Scholar
- Starkey Duncan. 1972. Some signals and rules for taking speaking turns in conversations. Journal of Personality and Social Psychology 23, 2 (1972), 283--292.Google ScholarCross Ref
- Starkey Duncan. 1974. On the structure of speaker-auditor interaction during speaking turns. Language in Society 3 (1974), 161--180.Google ScholarCross Ref
- Sergio Escalera, Oriol Pujol, Petia Radeva, Jordi Vitria, and M. Teresa Anguera. 2010. Automatic detection of dominance and expected interest. EURASIP Journal on Advances in Signal Processing 2010 (2010). Google ScholarDigital Library
- Daniel Gatica-Perez. 2009. Automatic nonverbal analysis of social interaction in small groups: A review. Image and Vision Computing 27, 12 (2009), 1775--1787. Google ScholarDigital Library
- Gerald Goetsch and David McFarland. 1980. Models of the distribution of acts in small discussion groups. Social Psychology Quarterly 43 (1980), 173--183.Google ScholarCross Ref
- Erving Goffman. 1981. Forms of Talk. University of Pennsylvania Press, Philadelphia, PA.Google Scholar
- Judith A. Hall, Erik J. Coats, and Lavonia Smith LeBeau. 2005. Nonverbal behavior and the vertical dimension of social relations: A meta-analysis. Psychological Bulletin 131 (2005), 898--924.Google ScholarCross Ref
- August B. Hollingshead. 1975. Four Factor Index of Social Issues, Yale University Press.Google Scholar
- Hung-Hsuan Huang, Naoya Baba, and Yukiko Nakano. 2011. Making a virtual conversational agent be aware of the addressee of users’ utterances in multi-user conversation from nonverbal information. In Proceedings of the 13th International Conference on Multimodal Interaction (ICMI’11). 401--408. Google ScholarDigital Library
- Hayley Hung, Dinesh Babu Jayagopi, Sileye Ba, Jean-Marc Odobez, and Daniel Gatica-Perez. 2008. Investigating automatic dominance estimation in groups from visual attention and speaking activity. In Proceedings of the 10th International Conference on Multimodal Interface (ICMI’08). 233--236. Google ScholarDigital Library
- Dinesh Babu Jayagopi, Hayley Hung, Chuohao Yeo, and Daniel Gatica-Perez. 2009. Modeling dominance in group conversations from nonverbal activity cues. In IEEE Transactions on Audio, Speech, and Language Processing, Special Issue on Multimodal Processing for Speech-Based Interactions 17, 3 (2009), 501--513. Google ScholarDigital Library
- Dinesh Babu Jayagopi and Jean-Marc Odobez. 2013. Given that, should I respond? Contextual addressee estimation in multi-party human-robot interactions. In Proceedings of Human Robot Interaction (HRI’13). Google ScholarDigital Library
- Michael Katzenmaier, Rainer Stiefelhagen, and Tanja Schultz. 2004. Identifying the addressee in human-human-robot interactions based on head pose and speech. In Proceedings of the International Conference on Multimodal Interfaces (ICMI’04). 144--151. Google ScholarDigital Library
- Adam Kendon. 1967. Some functions of gaze direction in social interaction. Acta Psychologica 26 (1967), 22--63.Google ScholarCross Ref
- Mark L. Knapp and Judith A. Hall. 2010. Nonverbal Communication in Human Interaction, Wadsworth.Google Scholar
- Brent Lance and Stacy Marsella. 2010. Glances, glares, and glowering: How should a virtual human express emotion through gaze?” Autonomous Agents and Multi-Agent Systems 20, 1 (2010), 50--69. Google ScholarDigital Library
- Akinobu Lee, Tatsuya Kawahara, and Kiyohiro Shikano. 2001. Julius—An open source real-time large vocabulary recognition engine. In Proceedings of the European Conference on Speech Communication and Technology (EUROSPEECH). 1691--1694.Google Scholar
- Marianne Schmid Mast. 2002. Dominance as expressed and inferred through speaking time: A meta-analysis. Human Communication Research 28, 3 (2002), 420--450.Google Scholar
- Samer Al Moubayed, Jonas Beskow, Gabriel Skantze, and Björn Granström. 2012. Furhat: A back-projected human-like robot head for multiparty human-machine interaction. Cognitive Behavioural Systems, Lecture Notes in Computer Science, Vol. 7403, 114--130. Google ScholarDigital Library
- Samer Al Moubayed, Jens Edlund, and Jonas Beskow. 2012. Taming Mona Lisa: Communicating gaze faithfully in 2D and 3D facial projections. ACM Transactions on Interactive Intelligent Systems (TIIS) 1, 2 (2012), 1--25. Google ScholarDigital Library
- Bilge Mutlu, Toshiyuki Shiwa, Takayuki Kanda, Hiroshi Ishiguro, and Norihiro Hagita. 2009. Footing in human-robot conversations: How robots might shape participant roles using gaze cues. In Proceedings of the 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI'09). 61--68. Google ScholarDigital Library
- Kazuhiro Otsuka, Junji Yamato, Yoshinao Takemae, and Hiroshi Murase. 2006. Quantifying interpersonal influence in face-to-face conversations based on visual attention patterns. In Proceedings of CHI’06 Extended Abstracts on Human Factors in Computing Systems (CHI EA’06). ACM, New York, NY. Google ScholarDigital Library
- Rutger Rienks and Dirk Heylen. 2005. Dominance detection in meetings using easily obtainable features. In Proceedings of the 2nd Joint Workshop on Multimodal Interaction and Related Machine Learning Algorithms (MLMI’05), Edinburgh, Scotland. Google ScholarDigital Library
- Rutger Rienks, Dong Zhang, Daniel Gatica-Perez, and Wilfried Post. 2006. Detection and application of influence rankings in small group meetings. In Proceedings of the 8th International Conference on Multimodal Interfaces (ICMI’06). 257--264. Google ScholarDigital Library
- Dairazalia Sanchez-Cortes, Oya Aran, Dinesh Babu Jayagopi, Marianne Schmid Mast, and Daniel Gatica-Perez. 2013. Emergent leaders through looking and speaking: From audio-visual data to multimodal recognition. Journal on Multimodal User Interfaces 7, 1--2 (2013), 39--53.Google Scholar
- Dairazalia Sanchez-Cortes, Oya Aran, Marianne Schmid Mast, and Daniel Gatica-Perez. 2010. Identifying emergent leadership in small groups using nonverbal communicative cues. In Proceedings of the International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction (ICMI-MLMI’10). Google ScholarDigital Library
- Samira Sheikhi, Dinesh Babu Jayagopi, Vasil Khalidov, and Jean-Marc Odobez. 2013. Context aware addressee estimation for human robot interaction. In Proceedings of the 6th Workshop on Eye Gaze in Intelligent Human Machine Interaction: Gaze in Multimodal Interaction (GazeIn’13). Google ScholarDigital Library
- Jacques Terken, Irene Joris, and Linda De Valk. 2007. Multimodal cues for addressee-hood in triadic communication with a human information retrieval agent. In Proceedings of the International Conference on Multimodal Interfaces (ICMI’07). 94--101. Google ScholarDigital Library
- Kyle James Tusing and James Price Dillard. 2000. The sounds of dominance: Vocal precursors of perceived dominance during interpersonal influence. Human Communication Research 26, 1 (2000), 148--171.Google Scholar
- Roel Vertegaal. 1999. The GAZE groupware system: Mediating joint attention in multiparty communication and collaboration. In Proceedings of CHI 1999. 294--301. Google ScholarDigital Library
- Roel Vertegaal, Robert Slagter, Gerrit C. van der Veer, and Anton Nijholt. 2001. Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. In Proceedings of CHI 2001. 301--308. Google ScholarDigital Library
- Akiko Yamazaki, Keiichi Yamazaki, Yoshinori Kuno, Matthew Burdelski, Michie Kawashima, and Hideaki Kuzuoka. 2008. Precision timing in human-robot interaction: Coordination of head movement and utterance. In Proceedings of CHI 2008. 131--140. Google ScholarDigital Library
Index Terms
- Generating Robot Gaze on the Basis of Participation Roles and Dominance Estimation in Multiparty Interaction
Recommendations
Determining robot gaze according to participation roles in multiparty conversations
HAI '14: Proceedings of the second international conference on Human-agent interactionGaze is an important nonverbal feedback signal in multiparty face-to-face conversations. To build a robot that can convey the appropriate attentional behavior in human-robot multiparty conversations, this paper analyzes human attentional behaviors in ...
Controlling Robot's Gaze according to Participation Roles and Dominance in Multiparty Conversations
HRI'15 Extended Abstracts: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction Extended AbstractsA robot's gaze behaviors are indispensable in allowing the robot to participate in multiparty conversations. To build a robot that can convey appropriate attentional behavior in multiparty human- robot conversations, this study proposes robot head gaze ...
Estimating conversational dominance in multiparty interaction
ICMI '12: Proceedings of the 14th ACM international conference on Multimodal interactionIt is important for conversational agents that manage multiparty conversations to recognize the group dynamics existing among the users. This paper proposes a method for estimating the conversational dominance of participants in group interactions. ...
Comments