research-article

Open access

Senorita: A Chorded Keyboard for Sighted, Low Vision, and Blind Mobile Users

Authors:

Gulnar Rakhmetulla,

Ahmed Sabbir ArifAuthors Info & Claims

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Pages 1 - 13

https://doi.org/10.1145/3313831.3376576

Published: 23 April 2020 Publication History

All formats PDF

Abstract

Senorita is a novel two-thumb virtual chorded keyboard for mobile devices. It arranges the letters on eight keys in a single row by the bottom edge of the device based on letter frequencies and the anatomy of the thumbs. Unlike most chorded methods, it provides visual cues to perform the chording actions in sequence, instead of simultaneously, when the actions are unknown, facilitating "learning by doing". Its compact design leaves most of the screen available and its position near the edge accommodates eyes-free text entry. In a longitudinal study with a smartphone, Senorita yielded on average 14 wpm. In a short-term study with a tablet, it yielded on average 9.3 wpm. In the final longitudinal study, it yielded 3.7 wpm with blind users, surpassing their Qwerty performance. Low vision users yielded 5.8 wpm. Further, almost all users found Senorita effective, easy to learn, and wanted to keep use it.

Supplemental Material

MP4 File

Preview video

Download
7.55 MB

MP4 File

Supplemental video

Download
48.50 MB

References

[1]

Mohammad Ali Alamdar Yazdi, Ashkan Negahban, Lora Cavuoto, and Fadel M Megahed. 2019. Optimization of split keyboard design for touchscreen devices. International Journal of Human-Computer Interaction 35, 6 (2019), 468--477.

[2]

Maraim Alnfiai and Srinivas Sampalli. 2016. An evaluation of SingleTapBraille keyboard: a text entry method that utilizes braille patterns on touchscreen devices. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '16). ACM, NY, NY, USA, 161--169.

Digital Library

[3]

Ahmed Sabbir Arif and Ali Mazalek. 2016. WebTEM: a web application to record text entry metrics. In Proceedings of the 2016 ACM International Conference on Interactive Surfaces and Spaces (ISS '16). ACM, NY, NY, USA, 415--420.

Digital Library

[4]

Ahmed Sabbir Arif and Wolfgang Stuerzlinger. 2009. Analysis of text entry performance metrics. In 2009 IEEE Toronto International Conference Science and Technology for Humanity (TIC-STH). 100--105.

[5]

Ahmed Sabbir Arif and Wolfgang Stuerzlinger. 2014. User adaptation to a faulty unistroke-based text entry technique by switching to an alternative gesture set. In Proceedings of Graphics Interface 2014 (GI '14). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 183--192. http://dl.acm.org/citation.cfm?id=2619648.2619679

Digital Library

[6]

Shiri Azenkot and Shumin Zhai. 2012. Touch behavior with different postures on soft smartphone keyboards. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12). ACM, NY, NY, USA, 251--260.

Digital Library

[7]

Nikola Banovic, Koji Yatani, and Khai N. Truong. 2013. Escape-keyboard: a sight-free one-handed text entry method for mobile touch-screen devices. International Journal of Mobile Human Computer Interaction (IJMHCI) 5, 3 (2013), 42--61.

Digital Library

[8]

Xiaojun Bi, Ciprian Chelba, Tom Ouyang, Kurt Partridge, and Shumin Zhai. 2012. Bimanual gesture keyboard. In Proceedings of the 25th Annual ACM Symposium on User Interface Software and Technology (UIST '12). ACM, NY, NY, USA, 137--146.

Digital Library

[9]

Xiaojun Bi and Shumin Zhai. 2016. IJQwerty: what difference does one key change make? Gesture typing keyboard optimization bounded by one key position change from Qwerty. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, NY, NY, USA, 49--58.

Digital Library

[10]

Syed Masum Billah, Yu-Jung Ko, Vikas Ashok, Xiaojun Bi, and IV Ramakrishnan. 2019. Accessible gesture typing for non-visual text entry on smartphones. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, NY, NY, USA, 376:1--376:12.

Digital Library

[11]

Matthew N. Bonner, Jeremy T. Brudvik, Gregory D. Abowd, and W. Keith Edwards. 2010. No-look notes: accessible eyes-free multi-touch text entry. In International Conference on Pervasive Computing. Springer, 409--426.

[12]

Rony Aldhea Dwi Cahya, Anik Nur Handayani, and Aji Prasetya Wibawa. 2018. Mobile Braille Touch Application for Visually Impaired People using Double Diamond Approach. In MATEC Web of Conferences, Vol. 197. EDP Sciences, 15007.

[13]

John M. Carroll and Caroline Carrithers. 1984. Training wheels in a user interface. Commun. ACM 27, 8 (Aug. 1984), 800--806.

Digital Library

[14]

John M. Carroll and Mary Beth Rosson. 1986. Paradox of the active user. In Interfacing Thought: Cognitive Aspects of Human-Computer interaction (1 ed.), John M. Carroll (Ed.). MIT Press, Cambridge, MA, USA, Chapter 5, 80--111.

[15]

Li Deng and Xuedong Huang. 2004. Challenges in adopting speech recognition. Commun. ACM 47, 1 (2004), 69--75.

Digital Library

[16]

Android Developers. 2017. TextToSpeech. (2017). https://developer.android.com/reference/android/speech/tts/TextToSpeech.

[17]

Wenrui Diao, Xiangyu Liu, Zhe Zhou, and Kehuan Zhang. 2014. Your voice assistant is mine: how to abuse speakers to steal information and control your phone. In Proceedings of the 4th ACM Workshop on Security and Privacy in Smartphones & Mobile Devices (SPSM '14). ACM, NY, NY, USA, 63--74.

Digital Library

[18]

Andrew Fowler, Kurt Partridge, Ciprian Chelba, Xiaojun Bi, Tom Ouyang, and Shumin Zhai. 2015. Effects of language modeling and its personalization on touchscreen typing performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 649--658.

Digital Library

[19]

Mayank Goel, Alex Jansen, Travis Mandel, Shwetak N. Patel, and Jacob O. Wobbrock. 2013. ContextType: using hand posture information to improve mobile touch screen text entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 2795--2798.

Digital Library

[20]

Nathan Green, Jan Kruger, Chirag Faldu, and Robert St. Amant. 2004. A reduced Qwerty keyboard for mobile text entry. In CHI '04 Extended Abstracts on Human Factors in Computing Systems (CHI EA '04). ACM, NY, NY, USA, 1429--1432.

Digital Library

[21]

Anna-Maria Gueorguieva, Gulnar Rakhmetulla, and Ahmed Sabbir Arif. 2018. Enabling input on tiny/headless systems using morse code. (Oct. 2018). https://www.asarif.com/pub/Gueorguieva_CCBM2018_ Morse_Code.pdf

[22]

João Guerreiro, André Rodrigues, Kyle Montague, Tiago Guerreiro, Hugo Nicolau, and Daniel Goncalves. 2015. TabLETS get physical: non-visual text entry on tablet devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15. ACM Press, Seoul, Republic of Korea, 39--42.

Digital Library

[23]

Tiago Guerreiro, Paulo Lagoá, Hugo Nicolau, Daniel Gonçalves, and Joaquim A Jorge. 2008. From tapping to touching: making touch screens accessible to blind users. IEEE MultiMedia 15, 4 (2008), 48--50.

Digital Library

[24]

Lilit Hakobyan, Jo Lumsden, Dympna O'Sullivan, and Hannah Bartlett. 2013. Mobile assistive technologies for the visually impaired. Survey of ophthalmology 58, 6 (2013), 513--528.

[25]

Jianwei Lai, Dongsong Zhang, Sen Wang, Isil Doga Yakut Kilic, and Lina Zhou. 2019. ThumbStroke: a virtual keyboard in support of sight-free and one-handed text entry on touchscreen mobile devices. ACM Transactions on Management Information Systems (TMIS) 10, 3 (2019), 11.

[26]

Jianwei Lai, Dongsong Zhang, Sen Wang, Isil Yakut Kilic, and Lina Zhou. 2018. A thumb stroke-based virtual keyboard for sight-free text entry on touch-screen mobile phones. In Proceedings of the 51st Hawaii International Conference on System Sciences.

[27]

Jeffrey S. Leon. 2008. Frequency of Character Pairs in English Language Text, Codes and Cryptography. (2008). http://homepages.math.uic.edu/~leon/mcs425-s08/handouts/char_freq2.pdf.

[28]

Stefan Leuthold, Javier A Bargas-Avila, and Klaus Opwis. 2008. Beyond web content accessibility guidelines: Design of enhanced text user interfaces for blind internet users. International Journal of Human-Computer Studies 66, 4 (2008), 257--270.

Digital Library

[29]

Frank Chun Yat Li, Leah Findlater, and Khai N. Truong. 2013. Effects of hand drift while typing on touchscreens. In Proceedings of Graphics Interface 2013 (GI '13). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 95--98. http://dl.acm.org/citation.cfm?id=2532129.2532146

[30]

Frank Chun Yat Li, Richard T. Guy, Koji Yatani, and Khai N. Truong. 2011. The 1Line keyboard: a Qwerty layout in a single line. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (UIST '11). ACM, NY, NY, USA, 461--470.

Digital Library

[31]

Stan J. Liebowitz and Stephen E. Margolis. 1990. The fable of the keys. The Journal of Law and Economics 33, 1 (1990), 1--25.

[32]

I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase Sets for Evaluating Text Entry Techniques. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, NY, NY, USA, 754--755.

Digital Library

[33]

I. Scott MacKenzie and Kumiko Tanaka-Ishii. 2007. Text Entry Systems: Mobility, Accessibility, Universality. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.

Digital Library

[34]

I. Scott MacKenzie and Shawn X. Zhang. 1999. The Design and Evaluation of a High-performance Soft Keyboard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '99). ACM, NY, NY, USA, 25--31.

Digital Library

[35]

Sergio Mascetti, Cristian Bernareggi, and Matteo Belotti. 2012. TypeInBraille: quick eyes-free typing on smartphones. In International Conference on Computers for Handicapped Persons. Springer, 615--622.

Digital Library

[36]

National Federation of the Blind (NFB). 2009. The Braille literacy crisis in America. Technical Report. National Federation of the Blind (NFB), Baltimore, MD, USA. https://nfb.org/images/nfb/documents/pdf/braille_literacy_report_web.pdf.

[37]

João Oliveira, Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves. 2011. Blind people and mobile touch-based text-entry: acknowledging the need for different flavors. In The Proceedings of the 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS '11). ACM, NY, NY, USA, 179--186.

Digital Library

[38]

Antti Oulasvirta, Anna Reichel, Wenbin Li, Yan Zhang, Myroslav Bachynskyi, Keith Vertanen, and Per Ola Kristensson. 2013. Improving two-thumb text entry on touchscreen devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI'13). ACM, NY, NY, USA, 2765--2774.

Digital Library

[39]

Kyudong Park, Taedong Goh, and Hyo-Jeong So. 2014. Toward accessible mobile application design: developing mobile application accessibility guidelines for people with visual impairment. In Proceedings of HCI Korea (HCIK '15). Hanbit Media, Inc., South Korea, 31--38. http://dl.acm.org/citation.cfm?id=2729485.2729491

Digital Library

[40]

Keith B. Perry and Juan Pablo Hourcade. 2008. Evaluating one handed thumb tapping on mobile touchscreen devices. In Proceedings of Graphics Interface 2008 (GI '08). Canadian Information Processing Society, Toronto, Ont., Canada, Canada, 57--64. http://dl.acm.org/citation.cfm?id=1375714.1375725

Digital Library

[41]

Marco Romano, Luca Paolino, Genoveffa Tortora, and Giuliana Vitiello. 2014. The tap and slide keyboard: A new interaction method for mobile device text entry. International Journal of Human-Computer Interaction 30, 12 (2014), 935--945.

[42]

Anthony Savidis and Constantine Stephanidis. 2004. Unified user interface development: the software engineering of universally accessible interactions. Univers. Access Inf. Soc. 3, 3--4 (Oct. 2004), 165--193.

[43]

George S Snoddy. 1926. Learning and stability: a psychophysiological analysis of a case of motor learning with clinical applications. Journal of Applied Psychology 10, 1 (1926), 1.

[44]

Mads Soegaard. 2019. Accessibility: Usability for all. Interaction Design Foundation. (2019). https://www.interaction-design.org/literature/article/accessibility-usability-for-all.

[45]

R. William Soukoreff and I. Scott MacKenzie. 2003. Metrics for text entry research: an evaluation of MSD and KSPC, and a new unified error metric. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, NY, NY, USA, 113--120.

Digital Library

[46]

Caleb Southern, James Clawson, Brian Frey, Gregory Abowd, and Mario Romero. 2012. An evaluation of BrailleTouch: mobile touchscreen text entry for the visually impaired. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services. ACM, 317--326.

Digital Library

[47]

Thomas S Tullis and Jacqueline N Stetson. 2004. A comparison of questionnaires for assessing website usability. In Usability professional association conference, Vol. 1. Minneapolis, USA.

[48]

Keith Vertanen, Haythem Memmi, Justin Emge, Shyam Reyal, and Per Ola Kristensson. 2015. VelociTap: investigating fast mobile text entry using sentence-based decoding of touchscreen keyboard input. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 659--668.

Digital Library

[49]

Daniel Vogel and Patrick Baudisch. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, NY, NY, USA, 657--666.

Digital Library

[50]

World Health Organization (WHO). 2008. Change the definition of blindness. (2008), 5 pages. http://www.who.int/blindness/Change%20the% 20Definition%20of%20Blindness.pdf.

[51]

Daniel Wigdor and Ravin Balakrishnan. 2004. A comparison of consecutive and concurrent input text entry techniques for mobile phones. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, NY, NY, USA, 81--88.

Digital Library

[52]

Jacob O. Wobbrock, Brad A. Myers, and John A. Kembel. 2003. EdgeWrite: a stylus-based text entry method designed for high accuracy and stability of motion. In Proceedings of the 16th Annual ACM Symposium on User Interface Software and Technology (UIST '03). ACM, NY, NY, USA, 61--70.

Digital Library

[53]

Suwen Zhu, Tianyao Luo, Xiaojun Bi, and Shumin Zhai. 2018. Typing on an invisible keyboard. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, NY, NY, USA, Article 439, 13 pages.

Digital Library

Cited By

Voutsakelis GTzimos NKokkonis GKontogiannis S(2024)A Haptic Braille Keyboard Layout for Smartphone ApplicationsElectronics10.3390/electronics1312240813:12(2408)Online publication date: 20-Jun-2024
https://doi.org/10.3390/electronics13122408
Zhang DLi ZAshok VSeiple WRamakrishnan IBi X(2024)Accessible Gesture Typing on Smartphones for People with Low VisionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676447(1-11)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676447
Komninos AStefanis VGarofalakis J(2023)A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind PersonsMultimodal Technologies and Interaction10.3390/mti70200227:2(22)Online publication date: 17-Feb-2023
https://doi.org/10.3390/mti7020022
Show More Cited By

Index Terms

Senorita: A Chorded Keyboard for Sighted, Low Vision, and Blind Mobile Users
1. Human-centered computing
  1. Accessibility
    1. Accessibility technologies
    2. Empirical studies in accessibility
  2. Human computer interaction (HCI)
    1. Interaction techniques
      1. Text input
2. Social and professional topics
  1. Professional topics
    1. Computing profession
      1. Assistive technologies

Recommendations

PuffText: a voiceless and touchless text entry solution for mobile phones
ASSETS '13: Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility

This work proposes the use of a low-cost software based puff controlled spin keyboard for mobile phones as an alternative interaction technology for people with motor disabilities. It attempts to explore the processing of the audio from the microphone ...
Blind people and mobile touch-based text-entry: acknowledging the need for different flavors
ASSETS '11: The proceedings of the 13th international ACM SIGACCESS conference on Computers and accessibility

The emergence of touch-based mobile devices brought fresh and exciting possibilities. These came at the cost of a considerable number of novel challenges. They are particularly apparent with the blind population, as these devices lack tactile cues and ...
An empirical investigation of the situationally-induced impairments experienced by blind mobile device users
W4A '16: Proceedings of the 13th International Web for All Conference

In this paper, we describe a study specifically focusing on the situationally-induced impairments and disabilities (SIIDs) which individuals who are blind encounter when interacting with mobile devices. We conducted semi-structured interviews with eight ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

April 2020

10688 pages

ISBN:9781450367080

DOI:10.1145/3313831

General Chairs:
Regina Bernhaupt
Eindhoven University of Technology, Netherlands
,
Florian 'Floyd' Mueller
Monash University, Australia
,
David Verweij
Newcastle University, UK
,
Josh Andres
RMIT, Australia
,
Program Chairs:
Joanna McGrenere
University of British Columbia, Canada
,
Andy Cockburn
University of Canterbury, New Zealand
,
Ignacio Avellino
University of Maryland Baltimore County, USA
,
Alix Goguey
Grenoble Alpes University, France
,
Pernille Bjørn
University of Copenhagen, Denmark
,
Shengdong (Shen) Zhao
National University of Singapore, Singapore
,
Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
,
Rafal Kocielnik
University of Washington, USA

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 April 2020

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '20

Sponsor:

SIGCHI

CHI '20: CHI Conference on Human Factors in Computing Systems

April 25 - 30, 2020

HI, Honolulu, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
1,143
Total Downloads

Downloads (Last 12 months)197
Downloads (Last 6 weeks)32

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Voutsakelis GTzimos NKokkonis GKontogiannis S(2024)A Haptic Braille Keyboard Layout for Smartphone ApplicationsElectronics10.3390/electronics1312240813:12(2408)Online publication date: 20-Jun-2024
https://doi.org/10.3390/electronics13122408
Zhang DLi ZAshok VSeiple WRamakrishnan IBi X(2024)Accessible Gesture Typing on Smartphones for People with Low VisionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676447(1-11)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3654777.3676447
Komninos AStefanis VGarofalakis J(2023)A Review of Design and Evaluation Practices in Mobile Text Entry for Visually Impaired and Blind PersonsMultimodal Technologies and Interaction10.3390/mti70200227:2(22)Online publication date: 17-Feb-2023
https://doi.org/10.3390/mti7020022
Li ZYu CGu YShi Y(2023)ResType: Invisible and Adaptive Tablet Keyboard Leveraging Resting FingersProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581055(1-14)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581055
Rakhmetulla GArif A(2023)Crownboard: A One-Finger Crown-Based Smartwatch Keyboard for Users with Limited DexterityProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3580770(1-22)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3580770
Puka RŁebkowski PDuda J(2021)One Row Keyboard: The Concept of Designing a Common Layout for Physical and Virtual KeyboardsElectronics10.3390/electronics1006066310:6(663)Online publication date: 12-Mar-2021
https://doi.org/10.3390/electronics10060663
Stefanis VKomninos AGarofalakis J(2020)Challenges in Mobile Text Entry using Virtual Keyboards for Low-Vision UsersProceedings of the 19th International Conference on Mobile and Ubiquitous Multimedia10.1145/3428361.3428391(42-46)Online publication date: 22-Nov-2020
https://dl.acm.org/doi/10.1145/3428361.3428391
Sharif MRakhmetulla GArif AJorge JFerreira AMarquardt NChevalier F(2020)TapStr: A Tap and Stroke Reduced-Qwerty for SmartphonesCompanion Proceedings of the 2020 Conference on Interactive Surfaces and Spaces10.1145/3380867.3426208(47-50)Online publication date: 8-Nov-2020
https://dl.acm.org/doi/10.1145/3380867.3426208

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten