skip to main content
research-article

Development of a Remote Therapy Tool for Childhood Apraxia of Speech

Published: 14 November 2015 Publication History

Abstract

We present a multitier system for the remote administration of speech therapy to children with apraxia of speech. The system uses a client-server architecture model and facilitates task-oriented remote therapeutic training in both in-home and clinical settings. The system allows a speech language pathologist (SLP) to remotely assign speech production exercises to each child through a web interface and the child to practice these exercises in the form of a game on a mobile device. The mobile app records the child's utterances and streams them to a back-end server for automated scoring by a speech-analysis engine. The SLP can then review the individual recordings and the automated scores through a web interface, provide feedback to the child, and adapt the training program as needed. We have validated the system through a pilot study with children diagnosed with apraxia of speech, their parents, and SLPs. Here, we describe the overall client-server architecture, middleware tools used to build the system, speech-analysis tools for automatic scoring of utterances, and present results from a clinical study. Our results support the feasibility of the system as a complement to traditional face-to-face therapy through the use of mobile tools and automated speech analysis algorithms.

References

[1]
L. Anthony, Q. Brown, J. Nias, B. Tate, and S. Mohan. 2012. Interaction and recognition challenges in interpreting children's touch and gesture input on mobile devices. In ACM Conference on Interactive Tabletops and Surfaces. 225--234.
[2]
ArtikPix. Retrieved October 1, 2015 from http://rinnapps.com/artikpix/.
[3]
ASHA Ad Hoc Committee on Apraxia of Speech in Children. American Speech-Language-Hearing Association. 2007. Childhood apraxia of speech {Technical Report}. Available from www.asha.org/policy.
[4]
K. J. Ballard, D. A. Robin, P. McCabe, and J. McDonald. 2010. A treatment for dysprosody in childhood apraxia of speech. Journal of Speech, Language, and Hearing Research 53, 1227--1245.
[5]
K. J. Ballard, H. D. Smith, D. Paramatmuni, P. McCabe, D. G. Theodoros, and B. E. Murdoch. 2012. Amount of kinematic feedback affects learning of speech motor skills. Motor Control 16, 106--119.
[6]
P. Boersma. 2002. Praat, a system for doing phonetics by computer. Glot International 5, 341--345.
[7]
H. T. Bunnell, D. M. Yarrington, and J. B. Polikoff. 2000. STAR: Articulation training for young children. In International Conference on Spoken Language Processing. 85--88.
[8]
N. Chen and Q. He. 2007. Using nonlinear features in automatic English lexical stress detection. In International Conference on Computational Intelligence and Security Workshops, 2007 (CISW’07). 328--332.
[9]
G. A. Constantinescu, D. G. Theodoros, T. G. Russell, E. C. Ward, S. J. Wilson, and R. Wootton. 2010. Home-based speech treatment for Parkinson's disease delivered remotely: A case report. Journal of Telemedicine and Telecare 16, 100--4.
[10]
A. L. Delaney and R. D. Kent. 2004. Developmental profiles of children diagnosed with apraxia of speech. Presented at the Annual Convention of the American-Speech-Language-Hearing Association.
[11]
M. de Sá, L. Carriço, J. Faria, and I. Sá. 2012. Children psychotherapy with mobile devices. In Human-Computer Interaction: The Agency Perspective, Studies in Computational Intelligence, M. Zacarias and J. V. de Oliveira, eds. Springer, 85--109.
[12]
J. Fletcher. 2010. The prosody of speech: Timing and rhythm. In The Handbook of Phonetic Sciences (2nd ed.),W. J. Hardcastle, J. Laver, F. E. Gibbon, eds. Wiley, Hoboken, NJ, 521--602.
[13]
K. Forrest. 2003. Diagnostic criteria of developmental apraxia of speech used by clinical speech-language pathologists. American Journal of Speech-Language Pathology 12, 376--380.
[14]
J. Froehlich, J. Wobbrock, and S. Kane. 2007. Barrier pointing: Using physical edges to assist target acquisition on mobile device touch screens. In ACM SIGACCESS Conference on Computers and Accessibility. 19--26.
[15]
R. Gaines, C. Missiuna, M. Egan, and J. McLean. 2008. Educational outreach and collaborative care enhances physician's perceived knowledge about Developmental Coordination Disorder. BMC Health Services Research 8, 1--9.
[16]
A. Georgeadis, D. M. Brennan, L. N. Barker, and C. R. Baron. 2003. Telerehabilitation and its effect on story retelling by adults with neurogenic communication disorders. In Clinical Aphasiology Conference. 639--652.
[17]
A. M. Harrison, W.-K. Lo, X. Qian, and H. Meng. 2009. Implementation of an extended recognition network for mispronunciation detection and diagnosis in computer-assisted pronunciation training. In SLaTE. 45--48.
[18]
D. G. Jamieson, G. Kranjc, K. Yu, and W. E. Hodgetts. 2004. Speech intelligibility of young school-aged children in the presence of real-life classroom noise. Journal of the American Academy of Audiology 15, 7, 508--517.
[19]
Y.-J. Kim and M. C. Beutnagel. 2011. Automatic assessment of American English lexical stress using machine learning algorithms. In SLaTE. 93--96.
[20]
H. Kolles and W. Feiden. 1995. Computer-assisted speech recognition in diagnostic pathology. Development of the DragonDictate. Pathologe 16, 6, 439--442.
[21]
T. Kristjansson, S. Deligne, and P. Olsen. 2005. Voicing features for robust speech detection. Entropy 2, 3.
[22]
S. Kwon and S. S. Narayanan. 2002. Speaker change detection using a new weighted distance measure. In International Conference on Spoken Language Processing (ICSLP’02). 2537--2540.
[23]
K. Li, X. Qian, S. Kang, and H. Meng. 2013. Lexical stress detection for L2 English speech using deep belief networks. In INTERSPEECH. 1811--1815.
[24]
E. Maas, D. A. Robin, S. N. A. Hula, S. E. Freedman, G. Wulf, K. J. Ballard, et al. 2008. Principles of motor learning in treatment of motor speech disorders. American Journal of Speech-Language Pathology 17, 277--298.
[25]
A. Maier, T. Haderlein, F. Stelzle, E. Nöth, E. Nkenke, and F. Rosanowski, et al. 2010. Automatic speech recognition systems for the evaluation of voice and speech disorders in head and neck cancer. EURASIP Journal on Audio, Speech, and Music Processing 1, Article ID: 926951.
[26]
P. McCabe, A. G. Macdonald-D’Silva, L. J. van Rees, K. J. Ballard, and J. Arciuli. 2014. Orthographically sensitive treatment for dysprosody in children with Childhood Apraxia of Speech using ReST intervention. Developmental Neurorehabilitation 17, 137--145.
[27]
O. Mich. 2009. Evaluation of software tools with deaf children. In International ACM SIGACCESS Conference on Computers and Accessibility. 235--236.
[28]
J. Moore and M. Churchward. 2010. Moodle 1.9 Extension Development. Packt Publishing, Birmingham, UK.
[29]
R. J. Moran, R. B. Reilly, P. de Chazal, and P. D. Lacy. 2006. Telephony-based voice pathology assessment using automated speech analysis. IEEE Transactions on Biomedical Engineering 53, 468--477.
[30]
E. Murray, P. McCabe, and K. J. Ballard. 2012. A comparison of two treatments for childhood apraxia of speech: Methods and treatment protocol for a parallel group randomised control trial. BMC Pediatrics 12, 112.
[31]
D. Newbury and A. Monaco. 2010. Genetic advances in the study of speech and language disorders. Neuron 68, 309--320.
[32]
K. Newell, M. Carlton, and A. Antoniou. 1990. The interaction of criterion and feedback information in learning a drawing task. Journal of Motor Behavior 22, 536--552.
[33]
A. M. Oster, D. House, A. Protopapas, and A. Hatzis. 2002. Presentation of a new EU project for speech therapy: Ortho-Logo-Paedia. Presented at the Proceedings of TMH-QPSR, Fonetik.
[34]
A. Parnandi, V. Karappa, Y. Son, M. Shahin, J. McKechnie, K. Ballard, et al. 2013. Architecture of an automated therapy tool for childhood apraxia of speech. In 15th International ACM SIGACCESS Conference on Computers and Accessibility. 5.
[35]
M. A. Rahurkar, J. H. Hansen, J. Meyerhoff, G. Saviolakis, and M. Koenig. 2002. Frequency band analysis for stress detection using a teager energy operator based feature. In INTERSPEECH.
[36]
J. Rick, A. Harris, P. Marshall, R. Fleck, N. Yuill, and Y. Rogers. 2009. Children designing together on a multi-touch tabletop: an analysis of spatial orientation and user interactions. In Conference on Interaction Design and Children. 106--114.
[37]
S. Rvachew and F. Brosseau-Lapre. 2006. Speech perception intervention. In Interventions for Speech Sound Disorders in Children, S. McLeod, (ed.). Brookes Publishing, Baltimore, MD.
[38]
R. A. Schmidt and T. Lee. 2005. Motor Control and Learning, 4th ed. Human Kinetics, Champaign, IL.
[39]
M. A. Shahin, B. Ahmed, and K. J. Ballard. 2012. Automatic classification of unequal lexical stress patterns using machine learning algorithms. In 2012 IEEE Spoken Language Technology Workshop (SLT). 388--391.
[40]
K. Shobaki, J. P. Hosom, and R. A. Cole. 2000. The OGI kids’ speech corpus and recognizers. In International Conference on Spoken Language Processing.
[41]
L. D. Shriberg, T. F. Campbell, H. B. Karlsson, R. L. Brown, J. L. Mcsweeny, and C. J. Nadler. 2003. A diagnostic marker for childhood apraxia of speech: The lexical stress ratio. Clinical Linguistics & Phonetics 17, 549--574.
[42]
J. Tepperman and S. Narayanan. 2005. Automatic Syllable Stress Detection Using Prosodic Features for Pronunciation Evaluation of Language Learners. In ICASSP (1), 937--940.
[43]
T. K. Veale. 1999. Targeting temporal processing deficits through fast ForWord® language therapy with a new twist. Language, Speech, and Hearing Services in Schools 30, 353--362.
[44]
D. Vilozni, M. Barker, H. Jellouschek, G. Heimann, and H. Blau. 2001. An interactive computer-animated system (SpiroGame) facilitates spirometry in preschool children. American Journal of Respiratory and Critical Care Medicine 164, 2200--2205.
[45]
M. Waite, L. Cahill, D. Theodoros, S. Busuttin, and T. Russell. 2006. A pilot study of online assessment of childhood speech disorders. Journal of Telemedicine and Telecare 92--94.
[46]
A. Williams. 2006. Multiple oppositions intervention. In Interventions for Speech Sound Disorders in Children, A. L. Williams, S. McLeod, R. J. McCauley, et al. (eds.). Brookes Publishing, Baltimore, MD.
[47]
P. Williams and H. Stephens. 2010. Nuffield Centre Dyspraxia Programme. In Interventions for Speech Sound Disorders in Children, A. L. Williams, S. McLeod, R. J. McCauley, et al. (eds.). Brookes Publishing, Baltimore, MD.
[48]
Y. Wren, S. Roulstone, and A. L. Williams. 2006. Computer-Based Interventions. In Interventions for Speech Sound Disorders in Children, S. McLeod (ed.), Brookes Publishing, Baltimore, MD.
[49]
S.-C. Yin, R. Rose, O. Saz, and E. Lleida. 2009. A study of pronunciation verification in a speech therapy application. In IEEE International Conference on Acoustics, Speech and Signal Processing, 4609--4612.
[50]
S. J. Young, G. Evermann, M. J. F. Gales, T. Hain, D. Kershaw, and G. Moore, et al. 2006. The HTK Book, version 3.4. Cambridge University, Cambridge, UK.
[51]
E. Zwicker. 1961. Subdivision of the audible frequency range into critical bands (Frequenzgruppen). The Journal of the Acoustical Society of America 33, 248--248.

Cited By

View all
  • (2024)Exploring the Role of Machine Learning in Diagnosing and Treating Speech Disorders: A Systematic Literature ReviewPsychology Research and Behavior Management10.2147/PRBM.S460283Volume 17(2205-2232)Online publication date: May-2024
  • (2024)Speech-Language Pathologist Practice Changes in Telehealth Speech-Language Therapy For Rural ChildrenJournal of Clinical Practice in Speech-Language Pathology10.1080/22087168.2021.1237030423:1(2-9)Online publication date: 27-Mar-2024
  • (2024)AI-based automated speech therapy tools for persons with speech sound disorder: a systematic literature reviewSpeech, Language and Hearing10.1080/2050571X.2024.2359274(1-22)Online publication date: 3-Jun-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Accessible Computing
ACM Transactions on Accessible Computing  Volume 7, Issue 3
Special Issue (Part 2) of Papers from ASSETS 2013
November 2015
79 pages
ISSN:1936-7228
EISSN:1936-7236
DOI:10.1145/2836329
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 November 2015
Accepted: 01 May 2015
Revised: 01 March 2015
Received: 01 June 2014
Published in TACCESS Volume 7, Issue 3

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Childhood apraxia of speech
  2. automated speech analysis
  3. speech therapy

Qualifiers

  • Research-article
  • Research
  • Refereed

Funding Sources

  • Australian Research Council Future Fellowship Scheme
  • Qatar National Research Fund (a member of Qatar Foundation)

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)89
  • Downloads (Last 6 weeks)14
Reflects downloads up to 06 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Exploring the Role of Machine Learning in Diagnosing and Treating Speech Disorders: A Systematic Literature ReviewPsychology Research and Behavior Management10.2147/PRBM.S460283Volume 17(2205-2232)Online publication date: May-2024
  • (2024)Speech-Language Pathologist Practice Changes in Telehealth Speech-Language Therapy For Rural ChildrenJournal of Clinical Practice in Speech-Language Pathology10.1080/22087168.2021.1237030423:1(2-9)Online publication date: 27-Mar-2024
  • (2024)AI-based automated speech therapy tools for persons with speech sound disorder: a systematic literature reviewSpeech, Language and Hearing10.1080/2050571X.2024.2359274(1-22)Online publication date: 3-Jun-2024
  • (2024)Treatment for Childhood Apraxia of Speech: Past, Present, and FutureJournal of Speech, Language, and Hearing Research10.1044/2024_JSLHR-23-00233(1-26)Online publication date: 20-May-2024
  • (2023)Barriers and facilitators to paediatric caregivers’ participation in virtual speech, language, and hearing services: A scoping reviewDIGITAL HEALTH10.1177/205520762312166849Online publication date: 29-Nov-2023
  • (2023)Equitable access to speech practice for rural Australian children using the SayBananas! mobile gameInternational Journal of Speech-Language Pathology10.1080/17549507.2023.220505725:3(388-402)Online publication date: 25-May-2023
  • (2022)Home-Based Activities for Children with Speech Sound Disorders: Requirements for a Tangible User Interface for Internet of Things ArtefactsApplied Sciences10.3390/app1218897112:18(8971)Online publication date: 7-Sep-2022
  • (2022)Intervention Mapping of a Gamified Therapy Prescription App for Children With Disabilities: User-Centered Design ApproachJMIR Pediatrics and Parenting10.2196/345885:3(e34588)Online publication date: 9-Aug-2022
  • (2021)An Automated Lexical Stress Classification Tool for Assessing Dysprosody in Childhood Apraxia of SpeechBrain Sciences10.3390/brainsci1111140811:11(1408)Online publication date: 25-Oct-2021
  • (2021)Leaving the Butler Behind: The Future of Role Reproduction in CUIProceedings of the 3rd Conference on Conversational User Interfaces10.1145/3469595.3469606(1-4)Online publication date: 27-Jul-2021
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media