Improved Gazing Transition Patterns for Predicting Turn-Taking in Multiparty Conversation
Author: 
Li TianAuthors Info & Claims
Li TianAuthors Info & ClaimsPages 215 - 219
Abstract
Turn-taking is an important attribute of conversation. Non-verbal behavior is very important for analyzing the turn-taking in multi-party conversation. In this study, we focused on the gaze behavior and improved the framework for predicting turn by analyzing the gaze transition patterns in a four-participant conversation. We define a set of gaze labels for the changes in the gaze of the speaker and the listener, and then encode them into different patterns. We also give an effective predictive model based on Naïve Bayes classifier for predicting turn shift. Experimental results show that our framework performs better than previous works.
References
[1]
Li, T., J. Qi, and Z. Zhen . "Predicting Turn-Taking by Compact Gazing Transition Patterns in Multiparty Conversation." Pacific-Rim Symposium on Image and Video Technology Springer, Cham, 2018.
[2]
Traum, D.R.: A computational theory of grounding in natural language conversation. A Computational Theory of Grounding in Natural Language Conversation (1994)
[3]
Traum, D., Rickel, J.: Embodied agents for multi-party dialogue in immersive virtual worlds. In: Proceedings of the first international joint conference on Autonomous agents and multiagent systems: part 2. pp. 766–773 (2002)
[4]
Thrisson, K.R.: Natural Turn-Taking Needs No Manual: Computational Theory and Model, from Perception to Action. Springer Netherlands (2002)
[5]
Schlangen, D.: From reaction to prediction experiments with computational models of turn-taking. Proceedings of Interspeech 2006, Panel on Prosody of Dialogue Acts and Turn-Taking (2006)
[6]
Sacks, H., Jefferson, G.: A simplest systematics for the organization of turn-taking for conversation. Language 50(4), 696–735 (1974)
[7]
Raux, A., Eskenazi, M.: A finite-state turn-taking model for spoken dialog systems. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics. pp. 629–637 (2009)
[8]
Otsuka, K., Takemae, Y., Yamato, J.: A probabilistic inference of multiparty conversation structure based on markov-switching models of gaze patterns, head directions, and utterances. In: In Proc. of Int. Conf. on Multimodal Interfaces. pp.191–198 (2005)
[9]
Otsuka, K., Araki, S., Ishizuka, K., Fujimoto, M., Heinrich, M., Yamato, J.: A realtime multimodal system for analyzing group meetings by combining face pose tracking and speaker diarization. In Proc. of Int. Conf. on Multimodal Interfaces pp. 257–264 (2008)
[10]
Otsuka, K.: Conversational scene analysis. IEEE signal processing magazine (28),127–131 (2011)
[11]
M., W.J., L., K.M.: Turn-taking in conversations. Journal of Communication 25(2), 75–92 (1975)
[12]
Levow, G.A.: Turn-taking in mandarin dialogue: Interactions of tone and intonation. Proceedings of the Sighan Workshop (2005)
[13]
Laskowski, K., Edlund, J., Heldner, M.: A single-port non-parametric model of turn-taking in multi-party conversation. In: Acoustics, Speech, and Signal Processing, 1988. ICASSP-88., 1988 International Conference on. pp. 5600 – 5603
[14]
Kumano, S., Otsuka, K., Dan, M., Yamato, J.: Recognizing communicative facial expressions for discovering interpersonal emotions in group meetings. In: Proc. International Conference on Multimodal Interaction. pp. 99–106 (2009)
[15]
de Kok, I., Heylen, D.: Multimodal end-of-turn prediction in multi-party meetings. In: Proceedings of the 2009 International Conference on Multimodal Interfaces. pp.91–98. ICMI-MLMI ’09
[16]
Predicting Turn-Taking by Compact Gazing Transition. Kendon, A.: Some functions of gaze-direction in social interaction. Acta Psychologica 26(1), 22–63 (1967)
[17]
Jokinen, K., Harada, K., Nishida, M., Yamamoto, S.: Turn-alignment using eyegaze and speech in conversational interaction. In: Annual Conference of the International Speech Communication Association. pp. 2018–2021 (2010)
[18]
Ishii, R., Otsuka, K., Kumano, S., Matsuda, M., Yamato, J.: Predicting next speaker and timing from gaze transition patterns in multi-party meetings pp. 79–86 (2013)
[19]
Ishii, R., Kumano, S., Otsuka, K.: Predicting next speaker based on head movement in multi-party meetings. In: Acoustics, Speech and Signal Processing (ICASSP), 2015 IEEE International Conference on (2015)
[20]
Haberman, S.J.: The analysis of residuals in cross-classified tables. Biometrics 29, 205–220 (1973)
[21]
Gorga, S., Otsuka, K.: Conversation scene analysis based on dynamic bayesian network and image-based gaze detection. In: In Proc. of Int. Conf. on Multimodal Interfaces (2010)
[22]
Goodwin, C.: Restarts, pauses, and the achievement of a state of mutual gaze at urn beginning. Sociological Inquiry 50 (1980)
[23]
Gatica-Perez, D.: Analyzing group interactions in conversations: a review. In: Multisensor Fusion and Integration for Intelligent Systems, 2006 IEEE International Conference on. pp. 41 – 46 (2006)
[24]
Ferrer, L., Shriberg, E., Stolcke, A.: Is the speaker done yet? faster and more accurate end-of-utterance detection usingprosody. Proc Icslp p. 2002 (2002)
[25]
Duncan, S.: Some signals and rules for taking speaking turns in conversations. Journal of Personality and Social Psychology 23(2), 283–292 (1972)
[26]
Dielmann, A., Garau, G., Bourlard, H.: Floor holder detection and end of speaker turn prediction in meetings. In: International Conference on Speech and Language Processing, Interspeech (2010)
[27]
Dan, B., Horvitz, E.: Multiparty turn taking in situated dialog: study, lessons,and directions. Proceedings of Annual Meeting of the Special Interest Group in Discourse & Dialogue pp. 98–109 (2011)
[28]
Chen, L., Harper, M.P.: Multimodal floor control shift detection. In: In Proc. Of Int. Conf. on Multimodal Interfaces. pp. 15–22 (2009)
[29]
Bohus, D., Horvitz, E.: Decisions about turns in multiparty conversation: Fromperception to action. In: In Proc. of Int. Conf. on Multimodal Interfaces. pp. 153–160 (2011)
[30]
Zirui. Zhang and Jun. Cheng, "Multi-Camera Tracking Helmet System," Journal of Image and Graphics, Vol. 1, No. 2, pp. 76-79, June 2013.
[31]
Jorge Henrique Busatto Casagrande and Marcelo Ricardo Stemmer, "Abnormal Motion Analysis for Tracking-Based Approaches Using Region-Based Method with Mobile Grid," Journal of Image and Graphics, Vol. 2, No. 1, pp. 22-27, June 2014.
[32]
Karthik Dinesh and Sumana Gupta, "Video Stabilization, Camera Motion Pattern Recognition and Motion Tracking Using Spatiotemporal Regularity Flow," Journal of Image and Graphics, Vol. 2, No. 1, pp. 33-40, June 2014.
[33]
Saad A. Yaseen and Sreela Sasi, "Robust Algorithm for Object Detection and Tracking in a Dynamic Scene," Journal of Image and Graphics, Vol. 2, No. 1, pp. 41-45, June 2014.
[34]
Chin-Shiuh Shieh, Yong-Shixa Jhan, Yuan-Li Liu, Mong-Fong Horng, and Tsair-Fwu Lee, "Video Object Tracking with Heuristic Optimization Methods," Journal of Image and Graphics, Vol. 6, No. 2, pp. 95-99, December 2018.
Recommendations
Predicting Turn-Taking by Compact Gazing Transition Patterns in Multiparty Conversation
Image and Video TechnologyAbstractGaze behavior plays an important role for analyzing turn-taking in multiparty conversation. In this study, we propose a general and powerful model for predicting turn-taking by analyzing gaze transition patterns in four-participant conversation. ...
Grounding and turn-taking in multimodal multiparty conversation
HCI'13: Proceedings of the 15th international conference on Human-Computer Interaction: interaction modalities and techniques - Volume Part IVThis study explores the empirical basis for multimodal conversation control acts. Applying conversation analysis as an exploratory approach, we attempt to illuminate the control functions of paralinguistic behaviors in managing multiparty conversation. ...
Impact of video editing based on participants' gaze in multiparty conversation
CHI EA '04: CHI '04 Extended Abstracts on Human Factors in Computing SystemsThis paper presents a video cut editing rule based on participants' gaze for establishing video editing rules that can accurately and clearly convey the flow of conversation in multiparty conversations to viewers. Demand is growing to be able to ...
Comments
Information & Contributors
Information
Published In
December 2021
219 pages
ISBN:9781450385893
DOI:10.1145/3511176
Copyright © 2021 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 12 March 2022
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
ICVIP 2021
ICVIP 2021: 2021 The 5th International Conference on Video and Image Processing
December 22 - 25, 2021
CA, Hayward, USA
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 85Total Downloads
- Downloads (Last 12 months)20
- Downloads (Last 6 weeks)1
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format