skip to main content
10.1145/2823513.2823517acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
short-paper

Automatic Speaker Identification from Interpersonal Synchrony of Body Motion Behavioral Patterns in Multi-Person Videos

Published: 13 November 2015 Publication History

Abstract

Interpersonal synchrony, i.e. the temporal coordination of persons during social interactions, was traditionally studied by developmental psychologists. It now holds an important role in fields such as social signal processing, usually treated as a dyadic issue. In this paper, we focus on the behavioral patterns from body motion to identify subtle social interactions in the context of multi-person discussion panels, typically involving more than two interacting individuals. We propose a computer-vision based approach for automatic speaker identification that takes advantage of body motion interpersonal synchrony between participants. The approach characterizes human body motion with a novel feature descriptor based on the pixel change history of multiple body regions, which is then used to classify the motor behavioral patterns of the participants into speaking/non-speaking. Our approach was evaluated on a challenging dataset of video segments from discussion panel scenes collected from YouTube. Results are very promising and suggest that interpersonal synchrony of motion behavior is indeed indicative of speaker/listener roles.

References

[1]
Delaherche, E., et al. 2012. Interpersonal synchrony: A survey of evaluation methods across disciplines. IEEE Trans. Affect. Comput. 3, 3, 349--365.
[2]
Sun, X., et al. 2011. Towards visual and vocal mimicry recognition in human-human interactions. In Proc. IEEE Int. Conf. Syst. Man Cybern. (SMC'11). IEEE, 367--373.
[3]
Rienks, R., et al. 2010. Differences in head orientation behavior for speakers and listeners: An experiment in a virtual environment. ACM Trans. Appl. Percept. 7, 1, 2:1--2:13.
[4]
Campbell, N. 2008. Multimodal processing of discourse information; the effect of synchrony. In Proc. 2nd Int. Symp. Univers. Commun. (ISUC'08). IEEE, 12--15.
[5]
Xiang, T., et al. 2002. Autonomous visual events detection and classification without explicit object-centred segmentation and tracking. In Proc. 13th Br. Mach. Vis. Conf. (BMVC'02). 233--242.
[6]
Viola, P., and Jones, M. 2001. Rapid object detection using a boosted cascade of simple features. In Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit. (CVPR'01). IEEE, I-511-I-518.
[7]
Lienhart, R., and Maydt, J. 2002. An extended set of Haar-like features for rapid object detection. In Proc. Int. Conf. Image Process. (ICIP'02). IEEE, I-900-I-903.
[8]
http://opencv.org/
[9]
http://www.anatomy4sculptors.com/hpc/
[10]
KaewTraKulPong, P., and Bowden, R. (2001). An improved adaptive background mixture model for real-time tracking with shadow detection. In Proc. 2nd Eur. Workshop Adv. Video Based Surveill. Syst. (AVBS'01). Kluwer Academic, 135--144.
[11]
Cortes, C., and Vapnik, V. 1995. Support-vector networks. Mach. Learn. 20, 3, 273--297.
[12]
Hsu, C. W., et al. 2003. A practical guide to support vector classification. Technical Report. National Taiwan University.
[13]
http://www.ffmpeg.org/

Cited By

View all
  • (2016)Look who is not talking: Assessing engagement levels in panel conversations2016 23rd International Conference on Pattern Recognition (ICPR)10.1109/ICPR.2016.7899947(2109-2114)Online publication date: Dec-2016

Index Terms

  1. Automatic Speaker Identification from Interpersonal Synchrony of Body Motion Behavioral Patterns in Multi-Person Videos

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          INTERPERSONAL '15: Proceedings of the 1st Workshop on Modeling INTERPERsonal SynchrONy And infLuence
          November 2015
          54 pages
          ISBN:9781450339865
          DOI:10.1145/2823513
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 13 November 2015

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. behavioral patterns
          2. body motion
          3. interpersonal synchrony
          4. pixel change history
          5. speaker identification
          6. support vector machines

          Qualifiers

          • Short-paper

          Funding Sources

          Conference

          ICMI '15
          Sponsor:
          ICMI '15: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
          November 13, 2015
          Washington, Seattle, USA

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)6
          • Downloads (Last 6 weeks)0
          Reflects downloads up to 16 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2016)Look who is not talking: Assessing engagement levels in panel conversations2016 23rd International Conference on Pattern Recognition (ICPR)10.1109/ICPR.2016.7899947(2109-2114)Online publication date: Dec-2016

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media