Abstract
Existing speech technology tends to be poorly suited for young children at play, both because of their age-specific pronunciation and because they tend to play together, making overlapping speech and side discussions about the play itself ubiquitous. We report the performance of an autonomous, multi-keyword spotter that has been trained and tested on data from a multi-player game designed to focus on these issues. In Mole Madness, children laugh, yell, speak at the same time, make side comments and even invent their own forms of keywords to control a virtual on-screen character. Within this challenging language environment, the system achieves 94 % overall recall and 85 % overall accuracy, providing child-child and child-robot pairs with responsive play in a rapid-paced game. This technology can enable others to create novel multi-party interactions for entertainment where a limited number of keywords has to be recognized.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Al Moubayed, S., Beskow, J., Skantze, G., Granström, B.: Furhat: a back-projected human-like robot head for multiparty human-machine interaction. In: Esposito, A., Esposito, A.M., Vinciarelli, A., Hoffmann, R., Müller, V.C. (eds.) Cognitive Behavioural Systems. LNCS, vol. 7403, pp. 114–130. Springer, Heidelberg (2012). doi:10.1007/978-3-642-34584-5_9
Bellegarda, J.R.: Spoken language understanding for natural interaction: the Siri experience. In: Mariani, J., Rosset, S., Garnier-Rizet, M., Devillers, L. (eds.) Natural Interaction with Robots, Knowbots and Smartphones, pp. 3–14. Springer, New York (2014)
Claypool, M., Claypool, K.: Latency and player actions in online games. Commun. ACM 49(11), 40–45 (2006)
Dewey, J.A., Carr, T.H.: When dyads act in parallel, a sense of agency for the auditory consequences depends on the order of the actions. Conscious. Cogn. 22(1), 155–166 (2013)
Edlund, J., Edelstam, F., Gustafson, J.: Human pause and resume behaviours for unobtrusive human like in-car spoken dialogue systems. In: EACL 2014, p. 73 (2014)
Gerosa, M., Giuliani, D., Narayanan, S., Potamianos, A.: A review of ASR technologies for children’s speech. In: Proceedings of the 2nd Workshop on Child, Computer and Interaction, WOCCI 2009, pp. 7:1–7:8. ACM, New York (2009)
Lehman, J., Al Moubayed, S.: Mole madness-a multi-child, fast-paced, speech-controlled game. In: AAAI Symposium on Turn-Taking and Coordination in Human-Machine Interaction, Stanford, CA (2015)
Liao, H., Pundak, G., Siohan, O., Carroll, M.K., Coccaro, N., Jiang, Q.M., Sainath, T.N., Senior, A., Beaufays, F., Bacchiani, M.: Large vocabulary automatic speech recognition for children. In: Sixteenth Annual Conference of the International Speech Communication Association (2015). research.google.com
Shivakumar, P.G., Potamianos, A., Lee, S., Narayanan, S.: Improving speech recognition for children using acoustic adaptation and pronunciation modeling. In: Proceedings of Workshop on Child Computer Interaction, September 2014
Skantze, G., Al Moubayed, S.: IrisTK: A statechart-based toolkit for multi-party face-to-face interaction. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI 2012, pp. 69–76. ACM, New York (2012)
Sundar, H., Lehman, J.F., Singh, R.: Keyword spotting in multi-player voice driven games for children. In: Sixteenth Annual Conference of the International Speech Communication Association (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing AG
About this paper
Cite this paper
Lehman, J.F., Wolfe, N., Pereira, A. (2016). Multi-party Language Interaction in a Fast-Paced Game Using Multi-keyword Spotting . In: Traum, D., Swartout, W., Khooshabeh, P., Kopp, S., Scherer, S., Leuski, A. (eds) Intelligent Virtual Agents. IVA 2016. Lecture Notes in Computer Science(), vol 10011. Springer, Cham. https://doi.org/10.1007/978-3-319-47665-0_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-47665-0_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-47664-3
Online ISBN: 978-3-319-47665-0
eBook Packages: Computer ScienceComputer Science (R0)