Abstract
The interaction challenges presented by touch-screen enabled devices for blind and visually impaired people has been addressed extensively in the literature. However, the emergence of wrist-worn devices with small screens, the so-called smartwatches, exacerbates such challenges. This paper presents an evaluation of text entry methods for smartwatches, covering three existing approaches—Voice Input and two QUERTY methods: GBoard, and A4Keyboard—and three Braille input methods proposed by the authors. Nine blind users and a low-vision participant composed sentences with the six methods. Speed and error rates are presented and preference data are analyzed, showing the approval of two Braille methods and a positive reception to one of the QWERTY methods, followed by the Voice Input. Each modality has its own set of strengths and weaknesses, which suggests room for improvement before the widespread adoption of smartwatch text entry as a reliable and accessible method for blind people.










Similar content being viewed by others
References
Abdolrahmani, A., Kuber, R., Hurst, A.: An empirical investigation of the situationally-induced impairments experienced by blind mobile device users. In: Proceedings of the 13th Web for All Conference, W4A ’16, pp. 21:1–21:8. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2899475.2899482. http://doi.acm.org/10.1145/2899475.2899482
Albanna, M.M.: Swiftbraille - connect the dots. (2017). http://en.swiftbraille.com/. [Last accessed December 24, 2018]
Alnfiai, M., Sampalli, S.: An evaluation of singletapbraille keyboard: a text entry method that utilizes braille patterns on touchscreen devices. In: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS ’16, pp. 161–169. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2982142.2982161
Alnfiai, M., Sampalli, S.: BrailleEnter: a touch screen Braille text entry method for the blind. Proc. Comput. Sci. 109, 257–264 (2017)
Azenkot, S., Lee, N.B.: Exploring the use of speech input by blind people on mobile devices. In: Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS ’13, pp. 11:1–11:8. ACM, New York, NY, USA (2013). https://doi.org/10.1145/2513383.2513440
Bardot, S., Serrano, M., Jouffrais, C.: From tactile to virtual: using a smartwatch to improve spatial map exploration for visually impaired users. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, MobileHCI ’16, pp. 100–111. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2935334.2935342
Billah, S.M., Ashok, V., Ramakrishnan, I.: Write-it-yourself with the aid of smartwatches: a wizard-of-Oz experiment with blind people. In: 23rd International Conference on Intelligent User Interfaces, IUI ’18, pp. 427–431. ACM, New York, NY, USA (2018). https://doi.org/10.1145/3172944.3173005
Billah, S..M., Ko, Y..J., Ashok, V., Bi, X., Ramakrishnan, I.: Accessible gesture typing for non-visual text entry on smartphones. Proc. 2019 CHI Conf. Human Fact. Comput. Syst. 2019, 1–12 (2019). https://doi.org/10.1145/3290605.3300606
Bonner, M.N., Brudvik, J.T., Abowd, G.D., Edwards, W.K.: No-look notes: accessible eyes-free multi-touch text entry. In: International Conference on Pervasive Computing, pp. 409–426. Springer (2010). https://doi.org/10.1007/978-3-642-12654-3_24
Britto Neto, L.d.S., Maike, V.R.M.L., Koch, F.L., Baranauskas, M.C.C., Rocha, A.d.R., Goldenstein, S.K.: A wearable face recognition system built into a smartwatch and the visually impaired user:. In: Proceedings of the 17th International Conference on Enterprise Information Systems, pp. 5–12. SCITEPRESS - Science and and Technology Publications, Barcelona, Spain (2015). https://doi.org/10.5220/0005370200050012
Cha, J.M., Choi, E., Lim, J.: Virtual sliding QWERTY: a new text entry method for smartwatches using Tap-N-Drag. Appl. Ergon. 51, 263–272 (2015)
Chen, X.A., Grossman, T., Fitzmaurice, G.: Swipeboard: a text entry technique for ultra-small interfaces that supports novice to expert transitions. In: Proceedings of the 27th annual ACM symposium on User interface software and technology, pp. 615–620. ACM Press (2014). https://doi.org/10.1145/2642918.2647354
Dalton, D.: Softbraillekeyboard (2016). https://docs.google.com/document/d/1OX6yaFjkK61VfROdU9uQNxjEZ_QNWxiMnUY_1ECnIyRQ/pub. [Last accessed December 24, 2018]
Dhar, A., Nittala, A., Yadav, K.: TactBack: VibroTactile Braille output using smartphone and smartwatch for visually impaired. In: Proceedings of the 13th Web for All Conference, W4A ’16, pp. 13:1–13:2. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2899475.2899514. http://doi.acm.org/10.1145/2899475.2899514
Dobosz, K., Szuścik, M.: OneHandBraille: An Alternative Virtual Keyboard for Blind People. In: Man-Machine Interactions 5, Advances in Intelligent Systems and Computing, pp. 62–71. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67792-7_7. https://link.springer.com/chapter/10.1007/978-3-319-67792-7_7
Frey, B., Rosier, K., Southern, C., Romero, M.: From texting app to Braille literacy. In: CHI ’12 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’12, pp. 2495–2500. ACM, New York, NY, USA (2012). https://doi.org/10.1145/2212776.2223825
Frey, B., Southern, C., Romero, M.: BrailleTouch: mobile texting for the visually impaired. In: C. Stephanidis (ed.) Universal Access in Human-Computer Interaction. Context Diversity, vol. 6767, pp. 19–25. Springer Berlin Heidelberg, Berlin, Heidelberg (2011). https://doi.org/10.1007/978-3-642-21666-4_3
Gaggi, O., Palazzi, C.E., Ciman, M., Bujari, A.: StepByWatch: a smartwatch-based enhanced navigation system for visually impaired users. In: 2018 15th IEEE Annual Consumer Communications & Networking Conference (CCNC), pp. 1–5. IEEE, Las Vegas, NV (2018). https://doi.org/10.1109/CCNC.2018.8319311
Gordon, M., Ouyang, T., Zhai, S.: WatchWriter: tap and gesture typing on a smartwatch miniature keyboard with statistical decoding. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 3817–3821. ACM Press (2016). https://doi.org/10.1145/2858036.2858242. http://dl.acm.org/citation.cfm?doid=2858036.2858242
Guerreiro, T., LagoÁ, P., Nicolau, H., Gonçalves, D., Jorge, J.A.: From tapping to touching: making touch screens accessible to blind users. IEEE MultiMedia 15(4), 48–50 (2008)
Gupta, A., Balakrishnan, R.: DualKey: miniature screen text entry via finger identification. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, pp. 59–70. ACM Press (2016). https://doi.org/10.1145/2858036.2858052. http://dl.acm.org/citation.cfm?doid=2858036.2858052
Hong, J., Heo, S., Isokoski, P., Lee, G.: SplitBoard: A Simple Split Soft Keyboard for Wristwatch-sized Touch Screens. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 1233–1236. ACM Press (2015). https://doi.org/10.1145/2702123.2702273. http://dl.acm.org/citation.cfm?doid=2702123.2702273
Hsiu, M.C., Huang, D.Y., Chen, C.A., Lin, Y.C., Hung, Y.p., Yang, D.N., Chen, M.: ForceBoard: using force as input technique on size-limited soft keyboard. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 599–604. ACM Press (2016). https://doi.org/10.1145/2957265.2961827. http://dl.acm.org/citation.cfm?doid=2957265.2961827
Inc., D.: Dot smartwatch (2018). https://dotincorp.com/. [Last accessed December 24, 2018]
Kane, S.K., Wobbrock, J.O., Ladner, R.E.: Usable gestures for blind people: understanding preference and performance. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI ’11, pp. 413–422. ACM, New York, NY, USA (2011). https://doi.org/10.1145/1978942.1979001. 00206
Lee, D., Lee, S., Yi, J., Kang, H., Oakley, I.: HexaBraille : touch-based text entry method on smart watches for blind users. International Conference on Ergonomics pp. 509–509 (2017). http://www.dbpia.co.kr/journal/articleDetail?nodeId=NODE07301939
Leporini, B., Buzzi, M.C., Buzzi, M.: Interacting with mobile devices via VoiceOver: usability and accessibility issues. In: Proceedings of the 24th Australian Computer-Human Interaction Conference on - OzCHI ’12, pp. 339–348. ACM Press, Melbourne, Australia (2012). https://doi.org/10.1145/2414536.2414591. http://dl.acm.org/citation.cfm?doid=2414536.2414591
Li, M., Fan, M., Truong, K.N.: BrailleSketch: a gesture-based text input method for people with visual impairments. In: Proceedings of the 19th International ACM SIGACCESS Conference on Computers and Accessibility - ASSETS ’17, pp. 12–21. ACM Press, Baltimore, Maryland, USA (2017). https://doi.org/10.1145/3132525.3132528. http://dl.acm.org/citation.cfm?doid=3132525.3132528
Luna, M.M.: Wearable braille input proposals on github (2019). https://github.com/pixellab-ufg/WearableBraille. [Last accessed April 11, 2021]
Luna, M.M., de M. N. Soares, F.A.A., Nascimento, H.A.D., Quigley, A.: Braille text entry on smartwatches: an evaluation of methods for composing the Braille cell. In: Proceedings of the ACM SIGCHI Symposium on Engineering Interactive Computing Systems, EICS ’19, pp. 19:1–19:6. ACM, New York, NY, USA (2019). https://doi.org/10.1145/3319499.3328233. http://doi.acm.org/10.1145/3319499.3328233
Luna, M..M., Soares, F..A..A..M..N., Nascimento, H..A..D., Siqueira, J., Souza, E..F., Nascimento, T..H., Costa, R..M.: Text entry on smartwatches: a systematic review of literature. 2018 IEEE 42nd Annual Computer Software and Applications Conference (COMPSAC) 02, 272–277 (2018). https://doi.org/10.1109/COMPSAC.2018.10242
MacKenzie, I.S., Soukoreff, R.W.: Phrase sets for evaluating text entry techniques. In: CHI ’03 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’03, pp. 754–755. ACM, New York, NY, USA (2003). https://doi.org/10.1145/765891.765971. http://doi.acm.org/10.1145/765891.765971
MacKenzie, I.S., Tanaka-Ishii, K.: Text Entry Systems: Mobility, Accessibility. Universality. Morgan Kaufmann, Burlington (2010)
Mascetti, S., Bernareggi, C., Belotti, M.: TypeInBraille: quick eyes-free typing on smartphones. In: Computers Helping People with Special Needs, Lecture Notes in Computer Science, pp. 615–622. Springer, Berlin, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31534-3_90. https://link.springer.com/chapter/10.1007/978-3-642-31534-3_90. 00023
Mattheiss, E., Regal, G., Schrammel, J., Garschall, M., Tscheligi, M.: Dots and letters: Accessible braille-based text input for visually impaired people on mobile touchscreen devices. In: K. Miesenberger, D. Fels, D. Archambault, P. Penaz, W. Zagler (eds.) Computers Helping People with Special Needs, Lecture Notes in Computer Science, vol. 8547, pp. 650–657. Springer International Publishing (2014). https://doi.org/10.1007/978-3-319-08596-8_100
Montague, K., Guerreiro, T.: Towards inviscid text-entry for blind people through non-visual word prediction interfaces. In: CHI Workshop on Inviscid Text-Entry and Beyond, p. 5 (2016)
Mottelson, A., Larsen, C., Lyderik, M., Strohmeier, P., Knibbe, J.: Invisiboard: maximizing display and input space with a full screen text entry method for smartwatches. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 53–59. ACM Press (2016). https://doi.org/10.1145/2935334.2935360. http://dl.acm.org/citation.cfm?doid=2935334.2935360
NASA, N.A.S.A.: The official nasa task load index (2019). https://humansystems.arc.nasa.gov/groups/TLX/. [Last accessed May 12, 2019]
Nascimento, T.H., Soares, F.A.A.M.N., Irani, P.P., Oliveira, L.L.G.d., Soares, A.d.S.: Method for text entry in smartwatches using continuous gesture recognition. In: 2017 IEEE 41st annual Computer software and applications conference (COMPSAC), pp. 549–554. IEEE (2017). https://doi.org/10.1109/COMPSAC.2017.168. http://ieeexplore.ieee.org/document/8029988/
Nicolau, H., Montague, K., Guerreiro, T., Guerreiro, J.a., Hanson, V.L.: B#: Chord-based correction for multitouch braille input. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1705–1708. ACM, New York, NY, USA (2014). https://doi.org/10.1145/2556288.2557269. http://doi.acm.org/10.1145/2556288.2557269
Oliveira, J., Guerreiro, T., Nicolau, H., Jorge, J., GonÇalves, D.: Brailletype: Unleashing braille over touch screen mobile phones. In: Proceedings of the 13th IFIP TC 13 International Conference on Human-computer Interaction - Volume Part I, INTERACT’11, pp. 100–107. Springer-Verlag, Berlin, Heidelberg (2011). http://dl.acm.org/citation.cfm?id=2042053.2042066
Oliveira, J.a., Guerreiro, T., Nicolau, H., Jorge, J., Gonçalves, D.: Blind people and mobile touch-based text-entry: Acknowledging the need for different flavors. In: The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility, ASSETS ’11, pp. 179–186. ACM, New York, NY, USA (2011). https://doi.org/10.1145/2049536.2049569. http://doi.acm.org/10.1145/2049536.2049569
Oney, S., Harrison, C., Ogan, A., Wiese, J.: ZoomBoard: a diminutive qwerty soft keyboard using iterative zooming for ultra-small devices. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, p. 2799. ACM Press (2013). https://doi.org/10.1145/2470654.2481387. http://dl.acm.org/citation.cfm?doid=2470654.2481387
Organization, W.H.: Visual impairment and blindness (2018). https://www.who.int/en/news-room/fact-sheets/detail/blindness-and-visual-impairment. [Last accessed 2019-04-14]
Pizza, S., Brown, B., McMillan, D., Lampinen, A.: Smartwatch in Vivo. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, CHI ’16, pp. 5456–5469. ACM, New York, NY, USA (2016). https://doi.org/10.1145/2858036.2858522. http://doi.acm.org/10.1145/2858036.2858522
Poirier, F., Belatar, M.: UniWatch: a soft keyboard for text entry on smartWatches using 3 keys. In: Kurosu, M. (ed.) Human-Computer Interaction. Interaction Platforms and Techniques, vol. 9732, pp. 341–349. Springer International Publishing, Cham (2016)
Quaresma, P.: Frequency analysis of the portuguese language (2008). https://www.mat.uc.pt/~pedro/cientificos/Cripto/CISUC-TR200803.pdf. [Last accessed January 22, 2020]
Šepić, B., Ghanem, A., Vogel, S.: BrailleEasy: One-handed Braille Keyboard for Smartphones. Stud. Health Technol. Inform. 217, 1030–1035 (2015)
Shao, Y.F., Chang-Ogimoto, M., Pointner, R., Lin, Y.C., Wu, C.T., Chen, M.: SwipeKey: a swipe-based keyboard design for smartwatches. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 60–71. ACM Press (2016). https://doi.org/10.1145/2935334.2935336. http://dl.acm.org/citation.cfm?doid=2935334.2935336
Shibata, T., Afergan, D., Kong, D., Yuksel, B.F., MacKenzie, I.S., Jacob, R.J.: DriftBoard: a panning-based text entry technique for ultra-small touchscreens. In: Proceedings of the 29th Annual Symposium on User Interface Software and Technology, pp. 575–582. ACM Press (2016). https://doi.org/10.1145/2984511.2984591. http://dl.acm.org/citation.cfm?doid=2984511.2984591
Siqueira, J., Soares, F.A.A.M.N., Ferreira, D.J., Silva, C.R.G., Berretta, L.d.O., Ferreira, C.B.R., Felix, I.M., Soares, A.d.S., Costa, R.M.d., Luna, M.M.: Braille text entry on smartphones: a systematic review of the literature. In: 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC), vol. 2, pp. 521–526 (2016). https://doi.org/10.1109/COMPSAC.2016.74. https://ieeexplore.ieee.org/document/7552266
Siqueira, J., Soares, F.A.A.M.N., Silva, C.R.G., Berretta, L.d.O., Ferreira, C.B.R., Felix, I.M., Luna, M.M.: Brailleécran: A braille approach to text entry on smartphones. In: 2016 IEEE 40th Annual Computer Software and Applications Conference (COMPSAC), vol. 2, pp. 608–609 (2016). https://doi.org/10.1109/COMPSAC.2016.5. https://ieeexplore.ieee.org/document/7552286
Soukoreff, R.W., MacKenzie, I.S.: Measuring errors in text entry tasks: An application of the levenshtein string distance statistic. In: CHI ’01 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’01, pp. 319–320. ACM, New York, NY, USA (2001). https://doi.org/10.1145/634067.634256. http://doi.acm.org/10.1145/634067.634256
Soukoreff, R.W., MacKenzie, I.S.: Minimum string distance (msd) download web page (2001). https://www.yorku.ca/mack/MSD/MSD.html. [Last accessed February 24, 2019]
Southern, C., Clawson, J., Frey, B., Abowd, G., Romero, M.: An evaluation of brailletouch: mobile touchscreen text entry for the visually impaired. In: Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services, pp. 317–326. ACM (2012)
Subash, N.S., Nambiar, S., Kumar, V.: BrailleKey: An alternative Braille text inputs system: Comparative study of an innovative simplified text input system for the visually impaired. In: 2012 4th International Conference on Intelligent Human Computer Interaction (IHCI), pp. 1–4 (2012). https://doi.org/10.1109/IHCI.2012.6481790
of The Blind, N.F.: The braille literacy crisisin america (2009). https://www.nfb.org/images/nfb/documents/pdf/braille_literacy_report_web.pdf. [Last accessed June 18, 2018]
Timmermann, J., Heuten, W., Boll, S.: Input methods for the Borg-RPE-scale on smartwatches. In: 2015 9th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), pp. 80–83 (2015). https://doi.org/10.4108/icst.pervasivehealth.2015.259220. https://eudl.eu/doi/10.4108/icst.pervasivehealth.2015.259220
Trindade, D., Rodrigues, A., Guerreiro, T., Nicolau, H.: Hybrid-Brailler: Combining physical and gestural interaction for mobile Braille input and editing. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI ’18. ACM Press, Montreal QC, Canada (2018). https://doi.org/10.1145/3173574.3173601. https://dl.acm.org/doi/10.1145/3173574.3173601
Turner, C.J., Chaparro, B.S., He, J.: Text input on a smartwatch QWERTY keyboard: tap vs. trace. Int. J. Human-Comput. Interact. (2016). https://doi.org/10.1080/10447318.2016.1223265
Ushida, K., Sekine, Y., Hasegawa, S.: IPPITSU: A one-stroke text entry method for touch panels using Braille system. In: 2014 IEEE 3rd Global Conference on Consumer Electronics (GCCE), pp. 374–375 (2014). https://doi.org/10.1109/GCCE.2014.7031257. https://ieeexplore.ieee.org/document/7031257
Vertanen, K., Memmi, H., Emge, J., Reyal, S., Kristensson, P.O.: VelociTap: investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 659–668. ACM Press (2015). https://doi.org/10.1145/2702123.2702135. http://dl.acm.org/citation.cfm?doid=2702123.2702135
Visuri, A., Sarsenbayeva, Z., van Berkel, N., Goncalves, J., Rawassizadeh, R., Kostakos, V., Ferreira, D.: Quantifying sources and types of smartwatch usage sessions. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, CHI ’17, pp. 3569–3581. ACM, New York, NY, USA (2017). https://doi.org/10.1145/3025453.3025817
Yeo, H.S., Lee, J., Bianchi, A., Quigley, A.: WatchMI: applications of watch movement input on unmodified smartwatches. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 594–598. ACM Press (2016). https://doi.org/10.1145/2957265.2961825. http://dl.acm.org/citation.cfm?doid=2957265.2961825
Yi, X., Yu, C., Xu, W., Bi, X., Shi, Y.: COMPASS: rotational keyboard on non-touch smartwatches. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, pp. 705–715. ACM Press (2017). https://doi.org/10.1145/3025453.3025454. http://dl.acm.org/citation.cfm?doid=3025453.3025454
Acknowledgments
The authors would like to thank our participants for the valuable contribution, dedication, patience and time. We hope our findings inspire work that help address your needs. Moreover, authors would like to thank CAPES (Coordenação de Aperfeiçoamento de Pessoal de Nível Superior—Brazil) [CAPES Finance Code #001] and [TED 7371 - CAPES # 3/2018 - Accessibility Tools -Ferramentas de acessibilidade] for the financial support.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
On behalf of all authors, the corresponding author states that there is no conflict of interest in this work.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
This work is dedicated to all the volunteers that helped developing and evaluating our ideas. We are extremely grateful for those who offered their time with such honesty and patience.
Rights and permissions
About this article
Cite this article
Luna, M.M., Nascimento, H.A.D., Quigley, A. et al. Text entry for the Blind on Smartwatches. Univ Access Inf Soc 22, 737–755 (2023). https://doi.org/10.1007/s10209-022-00870-2
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10209-022-00870-2