Skip to main content
SpringerLink
Log in

A Listening Keyboard for Users with Motor Impairments—A Usability Study

  • Published:
International Journal of Speech Technology Aims and scope Submit manuscript

Abstract

Computer users with motor impairments find it difficult and, in many cases, impossible to access PC functionality through the physical keyboard-and-mouse interface. Studies show that even able-bodied users experience similar difficulties when interacting with mobile devices; this is due to the reduced size/usability of the input interfaces. Advances in speech recognition have made it possible to design speech interfaces for alphanumeric data entry and indirect manipulation (cursor control). Although several related commercial applications exist, such systems do not provide a complete solution for arbitrary keyboard and mouse access, such as the access needed for, say, typing, compiling, and executing a C++ program.

We carried out a usability study to support the development of a speech user interface for arbitrary keyboard access and mouse control. The study showed that speech interaction with an ideal listening keyboard is better for users with motor impairments than handstick, in terms of task completion time (37% better), typing rate (74% better), and error rates (63% better). We believe that these results apply to both permanent and task-induced motor impairments. In particular, a follow-up experiment showed that handstick approximates conventional modes of alphanumeric input available on mobile devices (e.g., PDAs, cellular phones, and personal organizers). These modes of input include miniaturized keyboards, stylus “soft” keyboards, cellular phone numberpads, and handwriting recognition software. This result suggests that a listening keyboard would be an effective mode for alphanumeric input on future mobile devices.

This study contributed to the development of SUITEKeys—a speech user interface for arbitrary keyboard and mouse access available for MS platforms as freeware.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Calverley, B. (1999). Machine Demonstrates superhuman speech recognition abilities. USC News Service, news release 0999025. Also available at http://uscnews.usc.edu/newsreleases.

  • Christian, K., Kules, B., Shneiderman, B., and Youssef, A. (2000). A comparison of voice-controlled and mouse-controlled web browsing. Proceedings of the Fourth International ACM Conference on Assistive Technologies (ASSETS 2000). New York: ACM Press, pp. 72–79.

    Google Scholar 

  • Comerford, R. (1998). Pocket computers ignite OS battle. IEEE Spectrum, 35(5):43–48.

    Google Scholar 

  • Danis, C., Comerford, L., Janke, E., Davies, K., DeVries, J., and Bertrand, A. (1994). StoryWriter: A speech-oriented editor. Proceedings of Human Factors in Computing Systems (CHI '94)-Conference Companion New York: ACM Press, pp. 277–278.

    Google Scholar 

  • Fell, H.J., MacAuslan, J., Ferrier, L.J., and Chenausky, K. (1999). Automatic babble recognition for early detection of speech related disorders. Behaviour & Information Technology, 18(1):56–63.

    Google Scholar 

  • Goldstein, M., Book, R., Alsio, G., and Tessa, S. (1998). Ubiquitous input for wearable computing: QWERTY keyboard without a board. Proceedings of the First Workshop on Human Computer Interaction with Mobile Devices, Glasgow, Scotland. Available at www.dcs.gla.ac.uk/~johnson/papers/mobile/HCIMD1.html.

  • Gould, J.D., Conti, J., and Hovanyecz, T. (1983). Composing letters with a simulated listening typewriter. Communications of the ACM, 26(4): 295–308.

    Google Scholar 

  • Hofstadter, D.R. (1989). Gödel, Escher, Bach: An Eternal Golden Braid. New York: Vintage Books.

    Google Scholar 

  • Karl, L., Pettey, M., and Shneiderman, B. (1993). Speech-activated versus mouse-activated commands for word processing applications. International Journal of Man-Machine Studies, 39(4):667–687.

    Google Scholar 

  • Kelley, J.F. (1984). An iterative design methodology for user-friendly natural language office information applications. ACM Transactions on Office Information Systems, 2(1):26–41.

    Google Scholar 

  • Lazzaro, J.J. (2001). Speech-enabling applications. Byte Magazine, April 4, 2001. Also available at www.byte.com/column/ BYT20010404S0005.

  • Leggett, J. and Williams, G. (1984). An empirical investigation of voice as an input modality for computer programming. International Journal of Man-Machine Studies, 21(1):493–520.

    Google Scholar 

  • Levitt, H. (1994). Speech processing for physical and sensory disabilities. In D.B. Roe and J.G. Wilpon (Eds.), Voice Communication Between Humans and Machines. Washington, DC: National Academy of Sciences, pp. 311–343.

    Google Scholar 

  • MacKenzie, I.S., Zhang, S.X., and Soukoreff, R.W. (1999). Text entry using soft keyboards. Behaviour & Information Technology, 18(4):235–244.

    Google Scholar 

  • Malkewitz, R. (1998). Head pointing and speech control as a handsfree interface to desktop computing. Proceedings of The Third International ACM Conference on Assistive Technologies (ASSETS '98) New York: ACM Press, pp. 182–188.

    Google Scholar 

  • Manaris, B. and Dominick, W.D. (1993). NALIGE: A user interface management system for the development of natural language interfaces. International Journal of Man-Machine Studies, 38(6):891–921.

    Google Scholar 

  • Manaris, B. and Harkreader, A. (1997). SUITE: Speech understanding interface tools and environments. Proceedings of 10th International Florida AI Research Symposium (FLAIRS-97). Menlo Park, CA: AAAI Press, pp. 247–252.

    Google Scholar 

  • Manaris, B. and Harkreader, A. (1998). SUITEKeys: A speech understanding interface for the motor-control challenged. Proceedings of The Third International ACM Conference on Assistive Technologies (ASSETS '98). New York: ACM Press, pp. 108–115.

    Google Scholar 

  • Manaris, B., MacGyvers, V., and Lagoudakis, M. (1999). Universal access to mobile computing devices through speech input. Proceedings of 12th International Florida AI Research Symposium (FLAIRS-99). Menlo Park, CA: AAAI Press, pp. 286–92.

    Google Scholar 

  • Manaris, B., McCauley, R., and MacGyvers, V. (2001). An intelligent interface for keyboard and mouse control-Providing full access to PC functionality via speech. Proceedings of 14th International Florida AI Research Symposium (FLAIRS-01). Menlo Park, CA: AAAI Press, pp. 182–188.

    Google Scholar 

  • Markowitz, J.A. (1996). Using Speech Recognition. Upper Saddle River, NJ: Prentice Hall.

    Google Scholar 

  • McAlindon, P.J. and Staney, K.M. (1996). The Keybowl: An ergonomically designed document processing device. Proceedings of The Second International ACM Conference on Assistive Technologies (ASSETS '96). New York: ACM Press, pp. 86–93.

    Google Scholar 

  • Morrison, D.L., Green, T.R.G., Shaw, A.C., and Payne, S.J. (1984). Speech-controlled text-editing: Effects of input modality and of command structure. International Journal of Man-Machine Studies, 21(1):49–63.

    Google Scholar 

  • Mostow, J. and Aist, G. (1999). Reading and pronunciation tutor. U.S. Patent 5,920,838. Also see Mostow, J., Roth, S.F., Hauptmann, A.G., and Kane, M. (1994).Aprototype reading coach that listens. Proceedings of 12th National Conference on Artificial Intelligence (AAAI-94). Menlo Park, CA: AAAI Press, pp. 785–792.

    Google Scholar 

  • Murray, J.T., Van Praag, J., and Gilfoil, D. (1983). Voice versus keyboard control of cursor motion. Proceedings of the 27th Annual Meeting of Human Factors Society. Santa Monica, CA: Human Factors Society, p. 103. (cited in Shneiderman, 1997).

    Google Scholar 

  • Napier, H.A., Lane, D.M., Batsell, R.R., and Guadango, N.S. (1989). Impact of a restricted natural language interface on ease of learning and productivity. Communications of the ACM, 32(10):1190–1198.

    Google Scholar 

  • Pausch, R. and Leatherby, J.H. (1991). An empirical study: Adding voice input to a graphical editor. Journal of the American Voice Input/Output Society, 9(2):55–66.

    Google Scholar 

  • Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T. (1994). Human-Computer Interaction. Reading, MA: Addison Wesley.

  • Prentke Romich, Co. (1997). WiVik2 on-screen keyboard programs forWindows.Available atwww.prentrom.com/access/wivik.html.

  • Raman, T.V. (1996). Emacspeak-direct speech access. Proceedings of The Second International ACM Conference on Assistive Technologies (ASSETS '96). New York: ACM, Press, pp. 72–79.

    Google Scholar 

  • Roy, D. and Pentland, A. (1998). A phoneme probability display for individuals with hearing disabilities. Proceedings of The Third International ACM Conference on Assistive Technologies (ASSETS '98) New York: ACM Press, pp. 165–168.

    Google Scholar 

  • Schwartz, E. (2000). PDAs Learn to Listen Up. Info World, 22(6):1 and 10.

    Google Scholar 

  • Shneiderman, B. (1997). Designing the User Interface, 3rd ed. Reading, MA: Addison-Wesley.

  • Smith, A., Dunaway, J., Demasco, P., and Peischl, D. (1996). Multimodal input for computer access and augmentative communication. Proceedings of The Second International ACM Conference on Assistive Technologies (ASSETS '96). New York: ACM Press, pp. 80–85.

    Google Scholar 

  • Trewin, S. (1996). A study of input device manipulation difficulties. Proceedings of The Second International ACMConference on Assistive Technologies (ASSETS '96). New York: ACM Press, pp. 15–22.

    Google Scholar 

  • Weiser, M. (1994). Theworld is not a desktop. Interactions, 1(1):7–8.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, College of Charleston, 66 George Street, Charleston, SC, 29424, USA

    Bill Manaris

  2. Department of Psychology, University of Louisiana at Lafayette, P.O. Box 43131, Lafayette, LA, 70504, USA

    Valanne Macgyvers

  3. Department of Computer Science, Duke University, Box 90129, Durham, NC, 27708-0129, USA

    Michail Lagoudakis

Authors
  1. Bill Manaris
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Valanne Macgyvers
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Michail Lagoudakis
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Manaris, B., Macgyvers, V. & Lagoudakis, M. A Listening Keyboard for Users with Motor Impairments—A Usability Study. International Journal of Speech Technology 5, 371–388 (2002). https://doi.org/10.1023/A:1020917210165

Download citation

  • Issue Date:

  • DOI: https://doi.org/10.1023/A:1020917210165

Search

Navigation