Avinash Parnandi
Skip Abstract Section
Abstract
We present a multitier system for the remote administration of speech therapy to children with apraxia of speech. The system uses a client-server architecture model and facilitates task-oriented remote therapeutic training in both in-home and clinical settings. The system allows a speech language pathologist (SLP) to remotely assign speech production exercises to each child through a web interface and the child to practice these exercises in the form of a game on a mobile device. The mobile app records the child's utterances and streams them to a back-end server for automated scoring by a speech-analysis engine. The SLP can then review the individual recordings and the automated scores through a web interface, provide feedback to the child, and adapt the training program as needed. We have validated the system through a pilot study with children diagnosed with apraxia of speech, their parents, and SLPs. Here, we describe the overall client-server architecture, middleware tools used to build the system, speech-analysis tools for automatic scoring of utterances, and present results from a clinical study. Our results support the feasibility of the system as a complement to traditional face-to-face therapy through the use of mobile tools and automated speech analysis algorithms.
- L. Anthony, Q. Brown, J. Nias, B. Tate, and S. Mohan. 2012. Interaction and recognition challenges in interpreting children's touch and gesture input on mobile devices. In ACM Conference on Interactive Tabletops and Surfaces. 225--234. Google Scholar
Digital Library
- ArtikPix. Retrieved October 1, 2015 from http://rinnapps.com/artikpix/.Google Scholar
- ASHA Ad Hoc Committee on Apraxia of Speech in Children. American Speech-Language-Hearing Association. 2007. Childhood apraxia of speech {Technical Report}. Available from www.asha.org/policy.Google Scholar
- K. J. Ballard, D. A. Robin, P. McCabe, and J. McDonald. 2010. A treatment for dysprosody in childhood apraxia of speech. Journal of Speech, Language, and Hearing Research 53, 1227--1245.Google ScholarCross Ref
- K. J. Ballard, H. D. Smith, D. Paramatmuni, P. McCabe, D. G. Theodoros, and B. E. Murdoch. 2012. Amount of kinematic feedback affects learning of speech motor skills. Motor Control 16, 106--119.Google ScholarCross Ref
- P. Boersma. 2002. Praat, a system for doing phonetics by computer. Glot International 5, 341--345.Google Scholar
- H. T. Bunnell, D. M. Yarrington, and J. B. Polikoff. 2000. STAR: Articulation training for young children. In International Conference on Spoken Language Processing. 85--88.Google Scholar
- N. Chen and Q. He. 2007. Using nonlinear features in automatic English lexical stress detection. In International Conference on Computational Intelligence and Security Workshops, 2007 (CISW’07). 328--332. Google ScholarDigital Library
- G. A. Constantinescu, D. G. Theodoros, T. G. Russell, E. C. Ward, S. J. Wilson, and R. Wootton. 2010. Home-based speech treatment for Parkinson's disease delivered remotely: A case report. Journal of Telemedicine and Telecare 16, 100--4.Google ScholarCross Ref
- A. L. Delaney and R. D. Kent. 2004. Developmental profiles of children diagnosed with apraxia of speech. Presented at the Annual Convention of the American-Speech-Language-Hearing Association.Google Scholar
- M. de Sá, L. Carriço, J. Faria, and I. Sá. 2012. Children psychotherapy with mobile devices. In Human-Computer Interaction: The Agency Perspective, Studies in Computational Intelligence, M. Zacarias and J. V. de Oliveira, eds. Springer, 85--109.Google Scholar
- J. Fletcher. 2010. The prosody of speech: Timing and rhythm. In The Handbook of Phonetic Sciences (2nd ed.),W. J. Hardcastle, J. Laver, F. E. Gibbon, eds. Wiley, Hoboken, NJ, 521--602.Google Scholar
- K. Forrest. 2003. Diagnostic criteria of developmental apraxia of speech used by clinical speech-language pathologists. American Journal of Speech-Language Pathology 12, 376--380.Google ScholarCross Ref
- J. Froehlich, J. Wobbrock, and S. Kane. 2007. Barrier pointing: Using physical edges to assist target acquisition on mobile device touch screens. In ACM SIGACCESS Conference on Computers and Accessibility. 19--26. Google ScholarDigital Library
- R. Gaines, C. Missiuna, M. Egan, and J. McLean. 2008. Educational outreach and collaborative care enhances physician's perceived knowledge about Developmental Coordination Disorder. BMC Health Services Research 8, 1--9.Google ScholarCross Ref
- A. Georgeadis, D. M. Brennan, L. N. Barker, and C. R. Baron. 2003. Telerehabilitation and its effect on story retelling by adults with neurogenic communication disorders. In Clinical Aphasiology Conference. 639--652.Google Scholar
- A. M. Harrison, W.-K. Lo, X. Qian, and H. Meng. 2009. Implementation of an extended recognition network for mispronunciation detection and diagnosis in computer-assisted pronunciation training. In SLaTE. 45--48.Google Scholar
- D. G. Jamieson, G. Kranjc, K. Yu, and W. E. Hodgetts. 2004. Speech intelligibility of young school-aged children in the presence of real-life classroom noise. Journal of the American Academy of Audiology 15, 7, 508--517.Google ScholarCross Ref
- Y.-J. Kim and M. C. Beutnagel. 2011. Automatic assessment of American English lexical stress using machine learning algorithms. In SLaTE. 93--96.Google Scholar
- H. Kolles and W. Feiden. 1995. Computer-assisted speech recognition in diagnostic pathology. Development of the DragonDictate. Pathologe 16, 6, 439--442.Google Scholar
- T. Kristjansson, S. Deligne, and P. Olsen. 2005. Voicing features for robust speech detection. Entropy 2, 3.Google Scholar
- S. Kwon and S. S. Narayanan. 2002. Speaker change detection using a new weighted distance measure. In International Conference on Spoken Language Processing (ICSLP’02). 2537--2540.Google Scholar
- K. Li, X. Qian, S. Kang, and H. Meng. 2013. Lexical stress detection for L2 English speech using deep belief networks. In INTERSPEECH. 1811--1815.Google Scholar
- E. Maas, D. A. Robin, S. N. A. Hula, S. E. Freedman, G. Wulf, K. J. Ballard, et al. 2008. Principles of motor learning in treatment of motor speech disorders. American Journal of Speech-Language Pathology 17, 277--298.Google ScholarCross Ref
- A. Maier, T. Haderlein, F. Stelzle, E. Nöth, E. Nkenke, and F. Rosanowski, et al. 2010. Automatic speech recognition systems for the evaluation of voice and speech disorders in head and neck cancer. EURASIP Journal on Audio, Speech, and Music Processing 1, Article ID: 926951. Google ScholarDigital Library
- P. McCabe, A. G. Macdonald-D’Silva, L. J. van Rees, K. J. Ballard, and J. Arciuli. 2014. Orthographically sensitive treatment for dysprosody in children with Childhood Apraxia of Speech using ReST intervention. Developmental Neurorehabilitation 17, 137--145.Google ScholarCross Ref
- O. Mich. 2009. Evaluation of software tools with deaf children. In International ACM SIGACCESS Conference on Computers and Accessibility. 235--236. Google ScholarDigital Library
- J. Moore and M. Churchward. 2010. Moodle 1.9 Extension Development. Packt Publishing, Birmingham, UK. Google ScholarDigital Library
- R. J. Moran, R. B. Reilly, P. de Chazal, and P. D. Lacy. 2006. Telephony-based voice pathology assessment using automated speech analysis. IEEE Transactions on Biomedical Engineering 53, 468--477.Google ScholarCross Ref
- E. Murray, P. McCabe, and K. J. Ballard. 2012. A comparison of two treatments for childhood apraxia of speech: Methods and treatment protocol for a parallel group randomised control trial. BMC Pediatrics 12, 112.Google ScholarCross Ref
- D. Newbury and A. Monaco. 2010. Genetic advances in the study of speech and language disorders. Neuron 68, 309--320.Google ScholarCross Ref
- K. Newell, M. Carlton, and A. Antoniou. 1990. The interaction of criterion and feedback information in learning a drawing task. Journal of Motor Behavior 22, 536--552.Google ScholarCross Ref
- A. M. Oster, D. House, A. Protopapas, and A. Hatzis. 2002. Presentation of a new EU project for speech therapy: Ortho-Logo-Paedia. Presented at the Proceedings of TMH-QPSR, Fonetik.Google Scholar
- A. Parnandi, V. Karappa, Y. Son, M. Shahin, J. McKechnie, K. Ballard, et al. 2013. Architecture of an automated therapy tool for childhood apraxia of speech. In 15th International ACM SIGACCESS Conference on Computers and Accessibility. 5. Google ScholarDigital Library
- M. A. Rahurkar, J. H. Hansen, J. Meyerhoff, G. Saviolakis, and M. Koenig. 2002. Frequency band analysis for stress detection using a teager energy operator based feature. In INTERSPEECH.Google Scholar
- J. Rick, A. Harris, P. Marshall, R. Fleck, N. Yuill, and Y. Rogers. 2009. Children designing together on a multi-touch tabletop: an analysis of spatial orientation and user interactions. In Conference on Interaction Design and Children. 106--114. Google ScholarDigital Library
- S. Rvachew and F. Brosseau-Lapre. 2006. Speech perception intervention. In Interventions for Speech Sound Disorders in Children, S. McLeod, (ed.). Brookes Publishing, Baltimore, MD.Google Scholar
- R. A. Schmidt and T. Lee. 2005. Motor Control and Learning, 4th ed. Human Kinetics, Champaign, IL.Google Scholar
- M. A. Shahin, B. Ahmed, and K. J. Ballard. 2012. Automatic classification of unequal lexical stress patterns using machine learning algorithms. In 2012 IEEE Spoken Language Technology Workshop (SLT). 388--391.Google Scholar
- K. Shobaki, J. P. Hosom, and R. A. Cole. 2000. The OGI kids’ speech corpus and recognizers. In International Conference on Spoken Language Processing.Google Scholar
- L. D. Shriberg, T. F. Campbell, H. B. Karlsson, R. L. Brown, J. L. Mcsweeny, and C. J. Nadler. 2003. A diagnostic marker for childhood apraxia of speech: The lexical stress ratio. Clinical Linguistics & Phonetics 17, 549--574.Google ScholarCross Ref
- J. Tepperman and S. Narayanan. 2005. Automatic Syllable Stress Detection Using Prosodic Features for Pronunciation Evaluation of Language Learners. In ICASSP (1), 937--940.Google Scholar
- T. K. Veale. 1999. Targeting temporal processing deficits through fast ForWord® language therapy with a new twist. Language, Speech, and Hearing Services in Schools 30, 353--362.Google ScholarCross Ref
- D. Vilozni, M. Barker, H. Jellouschek, G. Heimann, and H. Blau. 2001. An interactive computer-animated system (SpiroGame) facilitates spirometry in preschool children. American Journal of Respiratory and Critical Care Medicine 164, 2200--2205.Google ScholarCross Ref
- M. Waite, L. Cahill, D. Theodoros, S. Busuttin, and T. Russell. 2006. A pilot study of online assessment of childhood speech disorders. Journal of Telemedicine and Telecare 92--94.Google ScholarCross Ref
- A. Williams. 2006. Multiple oppositions intervention. In Interventions for Speech Sound Disorders in Children, A. L. Williams, S. McLeod, R. J. McCauley, et al. (eds.). Brookes Publishing, Baltimore, MD.Google Scholar
- P. Williams and H. Stephens. 2010. Nuffield Centre Dyspraxia Programme. In Interventions for Speech Sound Disorders in Children, A. L. Williams, S. McLeod, R. J. McCauley, et al. (eds.). Brookes Publishing, Baltimore, MD.Google Scholar
- Y. Wren, S. Roulstone, and A. L. Williams. 2006. Computer-Based Interventions. In Interventions for Speech Sound Disorders in Children, S. McLeod (ed.), Brookes Publishing, Baltimore, MD.Google Scholar
- S.-C. Yin, R. Rose, O. Saz, and E. Lleida. 2009. A study of pronunciation verification in a speech therapy application. In IEEE International Conference on Acoustics, Speech and Signal Processing, 4609--4612. Google ScholarDigital Library
- S. J. Young, G. Evermann, M. J. F. Gales, T. Hain, D. Kershaw, and G. Moore, et al. 2006. The HTK Book, version 3.4. Cambridge University, Cambridge, UK.Google Scholar
- E. Zwicker. 1961. Subdivision of the audible frequency range into critical bands (Frequenzgruppen). The Journal of the Acoustical Society of America 33, 248--248.Google ScholarCross Ref
Index Terms
- Development of a Remote Therapy Tool for Childhood Apraxia of Speech
Recommendations
A Longitudinal Evaluation of Tablet-Based Child Speech Therapy with Apraxia World
Digital games can make speech therapy exercises more enjoyable for children and increase their motivation during therapy. However, many such games developed to date have not been designed for long-term use. To address this issue, we developed Apraxia ...
Architecture of an automated therapy tool for childhood apraxia of speech
ASSETS '13: Proceedings of the 15th International ACM SIGACCESS Conference on Computers and AccessibilityWe present a multi-tier system for the remote administration of speech therapy to children with apraxia of speech. The system uses a client-server architecture model and facilitates task-oriented remote therapeutic training in both in-home and clinical ...
Apraxia world: a speech therapy game for children with speech sound disorders
IDC '18: Proceedings of the 17th ACM Conference on Interaction Design and ChildrenThis paper presents Apraxia World, a remote therapy tool for speech sound disorders that integrates speech exercises into an engaging platformer-style game. In Apraxia World, the player controls the avatar with virtual buttons/joystick, whereas speech ...
Comments