Skip to main content
Springer Nature Link
Log in

Helping Students with Motor Impairments Program via Voice-Enabled Block-Based Programming

  • Conference paper
  • First Online:
Universal Access in Human-Computer Interaction. User and Context Diversity (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13309))

Included in the following conference series:

Abstract

Existing programming environments pose a challenge for students with upper-body motor impairments. This is because these environments require a level of dexterity that these students do not possess. For example, text-based programming environments require a lot of typing using a keyboard, while block-based programming environments require the use of a pointing device to drag and drop blocks of code. In our research, we aim to make the block-based programming environment Blockly, accessible to students with upper-body motor impairment, by adding speech as an alternative form of input. This voice-enabled version of Blockly will reduce the need for the use of a mouse or keyboard, hence making it more accessible. Our system consists of the original Blockly application, a speech recognition API, predefined voice commands, and a custom function. A preliminary study has been conducted. The results are encouraging, but they also revealed the need to broaden the target population, which was originally people with cerebral palsy, to people with any type of upper-body motor disability. Additionally, the study showed the need to redesign some voice commands. A prototype of the system has been implemented. As a next step, two additional studies will be conducted using this prototype, a usability study, and an A/B test.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://scratch.mit.edu/.

  2. 2.

    https://developers.google.com/blockly/.

  3. 3.

    https://developer.amazon.com/en-US/alexa.

  4. 4.

    https://www.apple.com/siri/.

  5. 5.

    https://github.com/google/blockly.

  6. 6.

    https://cmusphinx.github.io/.

  7. 7.

    https://cloud.google.com/speech-to-text.

  8. 8.

    https://www.nuance.com/dragon/for-developers/dragon-software-developer-kit.html.

  9. 9.

    https://developer.mozilla.org/en-US/docs/Web/API/Web_Speech_API.

  10. 10.

    https://www.google.com/intl/en/chrome/demos/speech.html.

References

  1. Ability connection. https://abilityconnection.org/. Accessed 13 Jan 2022

  2. Hour of Code: Hour of code activities. https://hourofcode.com/us/learn

  3. Code.org: Projects. https://studio.code.org/projects/public

  4. CSforALL: About csforall—csforall. https://www.csforall.org/about/. Accessed 13 Jan 2022

  5. CSforALL: Curriculum directory. https://www.csforall.org/projects_and_programs/curriculum_directory/. Accessed 13 Jan 2022

  6. Dai, L., Goldman, R., Sears, A., Lozier, J.: Speech-based cursor control: a study of grid-based solutions, pp. 94–101, January 2004. https://doi.org/10.1145/1028630.1028648

  7. Desilets, A.: Voicegrip: a tool for programming-by-voice. Int. J. Speech Technol. 4, June 2001. https://doi.org/10.1023/A:1011323308477

  8. Desilets, A., Fox, D., Norton, S.: Voicecode: an innovative speech interface for programming-by-voice, pp. 239–242, April 2006. https://doi.org/10.1145/1125451.1125502

  9. Centers for Disease Control, Prevention: What is cerebral palsy? https://www.cdc.gov/ncbddd/cp/facts.html. Accessed 13 Jan 2022

  10. The New York City Department of Education: Computer science for all. https://sites.google.com/strongschools.nyc/cs4all/

  11. Harada, S., Landay, J.A., Malkin, J., Li, X., Bilmes, J.A.: The vocal joystick: evaluation of voice-based cursor control techniques. In: Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility, Assets 2006, pp. 197–204. Association for Computing Machinery, New York (2006). https://doi.org/10.1145/1168987.1169021

  12. Harada, S., Wobbrock, J., Landay, J.: Voicedraw: a hands-free voice-driven drawing application for people with motor impairments, pp. 27–34, January 2007. https://doi.org/10.1145/1296843.1296850

  13. Humble, N.: Developing computational thinking skills in k-12 education through block programming tools. In: ICERI2019 Proceedings, pp. 4865–4873. 12th annual International Conference of Education, Research and Innovation, IATED, 11–13 November, 2019. https://doi.org/10.21125/iceri.2019.1190

  14. Lin, P., Van Brummelen, J., Lukin, G., Williams, R., Breazeal, C.: Zhorai: designing a conversational agent for children to explore machine learning concepts. Proceedings of the AAAI Conference on Artificial Intelligence 34, pp. 13381–13388, April 2020. https://doi.org/10.1609/aaai.v34i09.7061

  15. National Institute of Neurological Disorders, Stroke: Cerebral palsy: Hope through research. https://www.ninds.nih.gov/Disorders/Patient-Caregiver-Education/Hope-Through-Research/Cerebral-Palsy-Hope-Through-Research. Accessed 13 Jan 2022

  16. Oviatt, S.: Multimodal interactive maps: designing for human performance. Hum.-Comput. Interact. 12(1), 93–129 (1997). https://doi.org/10.1207/s15327051hci1201

  17. Rosenblatt, L., Carrington, P., Hara, K., Bigham, J.: Vocal programming for people with upper-body motor impairments, pp. 1–10, April 2018. https://doi.org/10.1145/3192714.3192821

  18. Sears, A., Lin, M., Karimullah, A.S.: Speech-based cursor control: understanding the effects of target size, cursor speed, and command selection 2, pp. 30–43 (2002). https://doi.org/10.1007/s10209-002-0034-6

  19. Society, I.N.: Motor impairment, April 2012. https://www.neuromodulation.com/motor-impairment. Accessed 12 Jan 2022

  20. Vertanen, K., MacKay, D.: Speech dasher: fast writing using speech and gaze. In: CHI (2010)

    Google Scholar 

  21. Wagner, A., Gray, J.: An empirical evaluation of a vocal user interface for programming by voice. Int. J. Inf. Technol. Syst. Approach 8, 47–63 (2015). https://doi.org/10.4018/IJITSA.2015070104

  22. Wagner, A., Gray, J.: An Empirical Evaluation of a Vocal User Interface for Programming by Voice, January 2017. https://doi.org/10.4018/978-1-5225-1759-7.ch012

  23. Wagner, A., Rudraraju, R., Datla, S., Banerjee, A., Sudame, M., Gray, J.: Programming by voice: a hands-free approach for motorically challenged children, May 2012. https://doi.org/10.1145/2212776.2223757

  24. Weintrop, D.: Block-based programming in computer science education. Commun. ACM 62(8), 22–25 (2019)

    Google Scholar 

  25. Weintrop, D., Wilensky, U.: To block or not to block, that is the question: Students’ perceptions of blocks-based programming. In: Proceedings of the 14th International Conference on Interaction Design and Children, IDC 2015, pp. 199–208. Association for Computing Machinery, New York (2015). https://doi.org/10.1145/2771839.2771860

  26. Whitehouse.gov: Computer science for all. https://obamawhitehouse.archives.gov/blog/2016/01/30/computer-science-all

  27. Wikipedia Contributors: Word error rate - Wikipedia, the free encyclopedia (2020). https://en.wikipedia.org/wiki/Word_error_rate. Accessed 10 Feb 2022

  28. Wikipedia contributors: Switch statement – Wikipedia, the free encyclopedia (2021). https://en.wikipedia.org/w/index.php?title=Switch_statement&oldid=1062689945. Accessed 14 Feb 2022)

  29. Williamson, J.: What i’ve learned about motor impairment, June 2017. http://simpleprimate.com/blog/motor. Accessed 12 Jan 2022

Download references

Author information

Authors and Affiliations

  1. University of North Texas, Denton, TX, 76205, USA

    Obianuju Okafor & Stephanie Ludi

Authors
  1. Obianuju Okafor
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephanie Ludi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Obianuju Okafor .

Editor information

Editors and Affiliations

  1. Foundation for Research and Technology - Hellas (FORTH), Heraklion, Crete, Greece

    Margherita Antona

  2. University of Crete and Foundation for Research and Technology - Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Okafor, O., Ludi, S. (2022). Helping Students with Motor Impairments Program via Voice-Enabled Block-Based Programming. In: Antona, M., Stephanidis, C. (eds) Universal Access in Human-Computer Interaction. User and Context Diversity. HCII 2022. Lecture Notes in Computer Science, vol 13309. Springer, Cham. https://doi.org/10.1007/978-3-031-05039-8_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05039-8_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05038-1

  • Online ISBN: 978-3-031-05039-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation