skip to main content
10.1145/2823513.2823518acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
short-paper

Exploring Children's Verbal and Acoustic Synchrony: Towards Promoting Engagement in Speech-Controlled Robot-Companion Games

Published: 13 November 2015 Publication History

Abstract

Children's interpersonal synchrony has been related to various benefits in social, mental and emotional development. We explore verbal and acoustic synchrony patterns between pairs of children playing a speech-controlled video game. Verbal features include word timing and duration patterns, while acoustic cues contain prosodic information. Synchrony is captured through a random-effects model taking into account multiple sources of variation and repeated measurements for each pair of children. Our findings indicate the presence of synchrony between participants during game play, which increases as they become more engaged in the game. These results are discussed in relation to personalized human-computer interaction and adaptive game environments.

References

[1]
S. Al Moubayed and J. Lehman. Toward better understanding of engagement in multiparty spoken interaction with children. ICMI, 2015.
[2]
E. Butler. Temporal interpersonal emotion systems: The "TIES" that form relationships. Personality and Social Psychology Review, 15(4):367--393, 2011.
[3]
E. Delaherche, M. Chetouani, A. Mahdhaoui, C. Saint-Georges, S. Viaux, and D. Cohen. Interpersonal synchrony: A survey of evaluation methods across disciplines. IEEE TAC, 3(3):349--365, 2012.
[4]
F. Eyben, M. Wöllmer, and B. Schuller. openSMILE: The Munich Versatile and Fast Open-Source Audio Feature Extractor. In Proc. ACM Multimedia, pp. 1459--1462, 2010.
[5]
A. Gelman and J. Hill. Data analysis using regression and multilevel/hierarchical models. Cambridge University Press, 2006.
[6]
A. Gonzales, J. Hancock, and J. Pennebaker. Language style matching as a predictor of social dynamics in small groups. Communication Research, 2009.
[7]
R. Gupta, C. Lee, S. Lee, and S. Narayanan. Assessment of a child's engagement using sequence model based features. In Proc. WASSS, 2013.
[8]
A. Harrist and R. Waugh. Dyadic synchrony: Its structure and function in children's development. Developmental Review, 22(4):555--592, 2002.
[9]
M. Hove and J. Risen. It's all in the timing: Interpersonal synchrony increases affiliation. Social Cognition, 27(6):949--960, 2009.
[10]
A. Karimi and Y. Lim. Children, engagement and enjoyment in digital narrative. In Proc. Ascilite, pp. 475--483, 2010.
[11]
M. Kehoe. Prosodic patterns in children's multisyllabic word productions. Language, Speech, and Hearing Services in Schools, 32(4):284--294, 2001.
[12]
A. Khalil, V. Minces, G. McLoughlin, and A. Chiba. Group rhythmic synchrony and attention in children. Frontiers in psychology, 4, 2013.
[13]
J. Lehman and S. Al Moubayed. Mole madness - a multi-child, fast-paced, speech-controlled game. In AAAI Symposium on Turn-taking and Coordination in Human-Machine Interaction, 2015.
[14]
J. Manson, G. Bryant, M. Gervais, and M. Kline. Convergence of speech rate in conversation predicts cooperation. Evolution and Human Behavior, 34(6):419--426, 2013.
[15]
T. Rabinowitch and A. Knafo-Noam. Synchronous rhythmic interaction enhances children's perceived similarity and closeness towards each other. PloS One, 10(4), 2015.
[16]
J. Read, S. Mac Farlane, and C. Casey. Endurability, engagement and expectations: Measuring children's fun. In Interaction design and children, 2:1--23, 2002.
[17]
J. Rehg, G. Abowd, A. Rozga, M. Romero, M. Clements, S. Sclaroff, I. Essa, O. Ousley, Y. Li, C. Kim, H. Rao, J. Kim, L. Presti, J. Zhang, D. Lantsman, J. Bidwell, and Z. Ye. Decoding children's social behavior. In Proc. CVPR, pp. 3414--3421, 2013.
[18]
M. Ronimus, J. Kujala, A. Tolvanen, and H. Lyytinen. Children's engagement during digital game-based learning of reading: The effects of time, rewards, and challenge. Computers & Education, 71:237--246, 2014.
[19]
N. Shaker, S. Asteriadis, G. N. Yannakakis, and K. Karpouzis. Fusing visual and behavioral cues for modeling user experience in games. IEEE Transactions on Cybernetics, 43(6):1519--1531, 2013.
[20]
C. Sidner, C. Lee, C. Kidd, N. Lesh, and C. Rich. Explorations in engagement for humans and robots. Artificial Intelligence, 166(1):140--164, 2005.
[21]
S. Woltering, V. Lishak, B. Elliott, L. Ferraro, and I. Granic. Dyadic attunement and physiological synchrony during mother-child interactions: An exploratory study in children with and without externalizing behavior problems. Journal of Psycho- pathology and Behavioral Assessment, 1--10, 2015.

Cited By

View all
  • (2022)Socially Interactive Agents in GamesThe Handbook on Socially Interactive Agents10.1145/3563659.3563675(493-526)Online publication date: 27-Oct-2022
  • (2022)Engagement in longitudinal child-robot language learning interactions: Disentangling robot and task engagementInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2022.10050133(100501)Online publication date: Sep-2022
  • (2022)The Handbook on Socially Interactive AgentsundefinedOnline publication date: 27-Oct-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
INTERPERSONAL '15: Proceedings of the 1st Workshop on Modeling INTERPERsonal SynchrONy And infLuence
November 2015
54 pages
ISBN:9781450339865
DOI:10.1145/2823513
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 November 2015

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. children
  2. engagement
  3. game design
  4. language
  5. speech
  6. synchrony

Qualifiers

  • Short-paper

Conference

ICMI '15
Sponsor:
ICMI '15: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION
November 13, 2015
Washington, Seattle, USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2022)Socially Interactive Agents in GamesThe Handbook on Socially Interactive Agents10.1145/3563659.3563675(493-526)Online publication date: 27-Oct-2022
  • (2022)Engagement in longitudinal child-robot language learning interactions: Disentangling robot and task engagementInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2022.10050133(100501)Online publication date: Sep-2022
  • (2022)The Handbook on Socially Interactive AgentsundefinedOnline publication date: 27-Oct-2022
  • (2021)A systematic review of physical–digital play technology and developmentally relevant child behaviourInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2021.10032330(100323)Online publication date: Dec-2021
  • (2020)Engagement in Human-Agent Interaction: An OverviewFrontiers in Robotics and AI10.3389/frobt.2020.000927Online publication date: 4-Aug-2020
  • (2019)Speech interface reformulations and voice assistant personification preferences of children and parentsInternational Journal of Child-Computer Interaction10.1016/j.ijcci.2019.04.00521:C(77-88)Online publication date: 1-Sep-2019
  • (2018)Children asking questionsProceedings of the 17th ACM Conference on Interaction Design and Children10.1145/3202185.3202207(300-312)Online publication date: 19-Jun-2018
  • (2017)Creating Prosodic Synchrony for a Robot Co-player in a Speech-controlled Game for ChildrenProceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction10.1145/2909824.3020244(91-99)Online publication date: 6-Mar-2017
  • (2016)Turn‐Taking, Children, and the Unpredictability of FunAI Magazine10.1609/aimag.v37i4.268537:4(55-62)Online publication date: 1-Dec-2016
  • (2015)Toward Better Understanding of Engagement in Multiparty Spoken Interaction with ChildrenProceedings of the 2015 ACM on International Conference on Multimodal Interaction10.1145/2818346.2820733(211-218)Online publication date: 9-Nov-2015

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media