skip to main content
10.1145/3517428.3550382acmconferencesArticle/Chapter ViewAbstractPublication PagesassetsConference Proceedingsconference-collections
poster

Voice-Enabled Blockly: Usability Impressions of a Speech-driven Block-based Programming System

Published: 22 October 2022 Publication History

Abstract

Block-based programming environments pose a challenge for people with upper-limb motor impairments. This is because they are highly dependent on the physical manipulation of a mouse or keyboard to drag and drop elements on the screen. Our research aims to make the block-based programming environment Blockly, accessible to users with upper limb motor impairments by adding voice as an alternative input modality. This voice-enabled version of Blockly will reduce the need for the use of a pointing device, thus increasing access for people with limited dexterity. The Voice-enabled Blockly system consists of the Blockly application, a speech recognition API, predefined voice commands, and a custom function. A usability study was conducted using a prototype of Voice-enabled Blockly. The results revealed that people with upper-limb motor impairments can use the system. However, it also exposed some shortcomings of the tool and gave some suggestions on how to fix them. Based on the findings, changes will be made to the system, and then, it will be evaluated in another user study in the near future.

References

[1]
Hadi Alathas. 2018. How to Measure Product Usability with the System Usability Scale (SUS) Score. Retrieved ”Apr 01, 2022” from https://uxplanet.org/how-to-measure-product-usability-with-the-system-usability-scale-sus-score-69f3875b858f
[2]
David Bau, Jeff Gray, Caitlin Kelleher, Josh Sheldon, and Franklyn Turbak. 2017. Learnable Programming: Blocks and Beyond. Commun. ACM 60, 6 (may 2017), 72–80. https://doi.org/10.1145/3015455
[3]
Georgios Fesakis and Kiriaki Serafeim. 2009. Influence of the Familiarization with ”Scratch” on Future Teachers’ Opinions and Attitudes about Programming and ICT in Education. SIGCSE Bull. 41, 3 (jul 2009), 258–262. https://doi.org/10.1145/1595496.1562957
[4]
Centers for Disease Control and Prevention. 2021. What is Cerebral Palsy?Retrieved ”Jan 13, 2022” from https://www.cdc.gov/ncbddd/cp/facts.html
[5]
N. Humble. 2019. Developing Computational Thinking Skills in K-12 Education Through Block Programming Tools. In ICERI2019 Proceedings (Seville, Spain) (12th annual International Conference of Education, Research and Innovation). IATED, 4865–4873. https://doi.org/10.21125/iceri.2019.1190
[6]
Surendheran Kaliyaperumal. 2019. Novice programmers’ attitude towards the introduction of block-based coding in Virtual Reality programming ABSTRACT. (06 2019).
[7]
Phoebe Lin, Jessica Van Brummelen, Galit Lukin, Randi Williams, and Cynthia Breazeal. 2020. Zhorai: Designing a Conversational Agent for Children to Explore Machine Learning Concepts. Proceedings of the AAAI Conference on Artificial Intelligence 34 (04 2020), 13381–13388. https://doi.org/10.1609/aaai.v34i09.7061
[8]
Stephanie Ludi and Mary Spencer. 2017. Design Considerations to Increase Block-based Language Accessibility for Blind Programmers Via Blockly. Journal of Visual Languages and Sentient Systems 3 (07 2017), 119–124. https://doi.org/10.18293/VLSS2017-013
[9]
National Institute of Neurological Disorders and Stroke. 2013. Cerebral Palsy: Hope Through Research. Retrieved ”Jan 13, 2022” from https://www.ninds.nih.gov/Disorders/Patient-Caregiver-Education/Hope-Through-Research/Cerebral-Palsy-Hope-Through-Research
[10]
Obianuju Okafor. 2022. Helping Students with Cerebral Palsy Program via Voice-Enabled Block-Based Programming. SIGACCESS Access. Comput.132, Article 2 (mar 2022), 1 pages. https://doi.org/10.1145/3523265.3523267
[11]
Obianuju Okafor and Ludi Stephanie. 2022. Helping Students with Upper-body Motor Disabilities Program via Voice-Enabled Block-based Programming(Lecture Notes in Computer Science (LNCS)). Springer, New York, NY, USA.
[12]
Lucas Rosenblatt, Patrick Carrington, Kotaro Hara, and Jeffrey Bigham. 2018. Vocal Programming for People with Upper-Body Motor Impairments. W4A ’18: Proceedings of the Internet of Accessible Things, 1–10. https://doi.org/10.1145/3192714.3192821
[13]
International Neuromodulation Society. 2012. Motor Impairment. https://www.neuromodulation.com/motor-impairment Accessed on 2022-01-12.
[14]
Will T. 2017. Measuring and Interpreting System Usability Scale (SUS) - UIUX Trend. Retrieved ”Mar 28, 2022” from https://uiuxtrend.com/measuring-system-usability-scale-sus/
[15]
Usability.gov. 2013. System Usability Scale (SUS). https://www.usability.gov/how-to-and-tools/methods/system-usability-scale.html [Online; accessed 28-March-2022].
[16]
Leticia Azucena Vaca-Cárdenas, Francesca Bertacchini, Assunta Tavernise, Lorella Gabriele, Antonella Valenti, Diana Elizabeth Olmedo, Pietro Pantano, and Eleonora Bilotta. 2015. Coding with Scratch: The design of an educational setting for Elementary pre-service teachers. In 2015 International Conference on Interactive Collaborative Learning (ICL). 1171–1177. https://doi.org/10.1109/ICL.2015.7318200
[17]
W3Schools. 2022. JavaScript Switch Statement. https://www.w3schools.com/js/js_switch.asp [Online; accessed 14-April-2022].
[18]
Amber Wagner and Jeff Gray. 2015. An Empirical Evaluation of a Vocal User Interface for Programming by Voice. Vol. 8. 47–63. https://doi.org/10.4018/IJITSA.2015070104
[19]
Amber Wagner and Jeff Gray. 2017. An Empirical Evaluation of a Vocal User Interface for Programming by Voice. https://doi.org/10.4018/978-1-5225-1759-7.ch012
[20]
Amber Wagner, Ramaraju Rudraraju, Srinivasa Datla, Avishek Banerjee, Mandar Sudame, and Jeff Gray. 2012. Programming by voice: a hands-free approach for motorically challenged children. (05 2012). https://doi.org/10.1145/2212776.2223757
[21]
David Weintrop. 2019. Block-Based Programming in Computer Science Education. Commun. ACM 62, 8 (July 2019), 22–25. https://doi.org/10.1145/3341221
[22]
David Weintrop and Uri Wilensky. 2015. To Block or Not to Block, That is the Question: Students’ Perceptions of Blocks-Based Programming. In Proceedings of the 14th International Conference on Interaction Design and Children (Boston, Massachusetts) (IDC ’15). Association for Computing Machinery, New York, NY, USA, 199–208. https://doi.org/10.1145/2771839.2771860
[23]
Joseph B. Wiggins, Fahmid M. Fahid, Andrew Emerson, Madeline Hinckle, Andy Smith, Kristy Elizabeth Boyer, Bradford Mott, Eric Wiebe, and James Lester. 2021. Exploring Novice Programmers’ Hint Requests in an Intelligent Block-Based Coding Environment. In Proceedings of the 52nd ACM Technical Symposium on Computer Science Education (Virtual Event, USA) (SIGCSE ’21). Association for Computing Machinery, New York, NY, USA, 52–58. https://doi.org/10.1145/3408877.3432538
[24]
James Williamson. 2017. What I’ve learned about motor impairment. http://simpleprimate.com/blog/motor Accessed on 2022-01-12.
[25]
Zhen Xu, Albert Ritzhaupt, Fengchun Tian, and Karthikeyan Umapathy. 2019. Block-based versus text-based programming environments on novice student learning outcomes: a meta-analysis study. Computer Science Education 29 (01 2019), 1–28. https://doi.org/10.1080/08993408.2019.1565233

Cited By

View all
  • (2024)Neurodiverse Programmers and the Accessibility of Parsons Problems: An Exploratory Multiple-Case StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630898(491-497)Online publication date: 7-Mar-2024
  • (2023)Programming by Voice: Exploring User Preferences and Speaking StylesProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597130(1-13)Online publication date: 19-Jul-2023
  • (2023)Voice vs Mouse: The Input Preference of People with Upper-Limb Impairments in a Multi-modal Block-Based Programming EnvironmentHCI International 2023 – Late Breaking Posters10.1007/978-3-031-49212-9_25(183-191)Online publication date: 12-Dec-2023
  • Show More Cited By

Index Terms

  1. Voice-Enabled Blockly: Usability Impressions of a Speech-driven Block-based Programming System

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      ASSETS '22: Proceedings of the 24th International ACM SIGACCESS Conference on Computers and Accessibility
      October 2022
      902 pages
      ISBN:9781450392587
      DOI:10.1145/3517428
      Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 October 2022

      Check for updates

      Author Tags

      1. Accessibility
      2. Blocked-based Programming
      3. Motor Impairments
      4. Speech Recognition
      5. Usability Testing

      Qualifiers

      • Poster
      • Research
      • Refereed limited

      Conference

      ASSETS '22
      Sponsor:

      Acceptance Rates

      ASSETS '22 Paper Acceptance Rate 35 of 132 submissions, 27%;
      Overall Acceptance Rate 436 of 1,556 submissions, 28%

      Upcoming Conference

      ASSETS '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)32
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 15 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Neurodiverse Programmers and the Accessibility of Parsons Problems: An Exploratory Multiple-Case StudyProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630898(491-497)Online publication date: 7-Mar-2024
      • (2023)Programming by Voice: Exploring User Preferences and Speaking StylesProceedings of the 5th International Conference on Conversational User Interfaces10.1145/3571884.3597130(1-13)Online publication date: 19-Jul-2023
      • (2023)Voice vs Mouse: The Input Preference of People with Upper-Limb Impairments in a Multi-modal Block-Based Programming EnvironmentHCI International 2023 – Late Breaking Posters10.1007/978-3-031-49212-9_25(183-191)Online publication date: 12-Dec-2023
      • (2022)Helping Students with Upper Limb Motor Impairments Program in a Block-Based Programming Environment Using Voiceundefined10.12794/metadc1985949Online publication date: Aug-2022

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media