skip to main content
10.1145/2470654.2470732acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Octopus: evaluating touchscreen keyboard correction and recognition algorithms via

Published: 27 April 2013 Publication History

Abstract

The time and labor demanded by a typical laboratory-based keyboard evaluation are limiting resources for algorithmic adjustment and optimization. We propose Remulation, a complementary method for evaluating touchscreen keyboard correction and recognition algorithms. It replicates prior user study data through real-time, on-device simulation. We have developed Octopus, a Remulation-based evaluation tool that enables keyboard developers to efficiently measure and inspect the impact of algorithmic changes without conducting resource-intensive user studies. It can also be used to evaluate third-party keyboards in a "black box" fashion, without access to their algorithms or source code. Octopus can evaluate both touch keyboards and word-gesture keyboards. Two empirical examples show that Remulation can efficiently and effectively measure many aspects of touch screen keyboards at both macro and micro levels. Additionally, we contribute two new metrics to measure keyboard accuracy at the word level: the Ratio of Error Reduction (RER) and the Word Score.

Supplementary Material

suppl.mov (chi2191-file3.mp4)
Supplemental video

References

[1]
Andrew, C. W., Maier, V., Green, P. (2004). From WER and RIL to MER and WIL: improved evaluation measures for connected speech recognition. Proc. of INTERSPEECH. 4 pages.
[2]
Android 4.2 http://www.android. com/whatsnew/
[3]
Android Developers. Tools: Andorid Debug Bridge. Accessed April 13, 2012. http://developer.android.com/guide/developing/tools/a db.html.
[4]
Azenkot, S. and Zhai, S. (2012). Touch Behavior with Different Postures on Soft Smart Phone Keyboards. Proc. of MobileHCI'12. 251--260.
[5]
Bi, X., Chelba, C., Ouyang, T., Partridge, K., and Zhai, S. (2012). Bimanual gesture keyboard. Proc. of UIST'12. 137--146.
[6]
Damn You, Auto-Correct! http://www.damnyouauto correct.com/. Accessed April 13, 2012.
[7]
Findlater, L., and Wobbrock, J.O. (2012). Personalized input: improving ten-finger touchscreen typing through automatic adaptation. Proc. of CHI '12. 815--824.
[8]
Goodman, J., Venolia, G., Steury, K., and Parker, C. (2002). Language modeling for soft keyboards. Proc. AAAI \'1802, Menlo Park, CA, USA, 419--424.
[9]
Gunawardana, A., Paek, T., and Meek, C. (2010). Usability guided key-target resizing for soft keyboards. Proc. IUI '10. ACM, New York, NY, USA, 111--118.
[10]
Kristensson, P.O. and Zhai, S. 2004. SHARK: a large vocabulary shorthand writing system for pen-based computers. Proc. of UIST \'1804. ACM Press: 43--52.
[11]
Lewis, J. R. (1992). Typing-key layouts for singlefinger or stylus input: initial user preference and performance (Technical Report No. 54729). Boca Raton, FL: International Business Machines Corporation.
[12]
Lewis, J. R., Kennedy, P. J., & LaLomia, M. J. (1999). Development of a Digram-Based Typing Key Layout for Single-Finger/Stylus Input. Proceedings of The Human Factors and Ergonomics Society 43rd Annual Meeting.
[13]
Lewis, J. R., LaLomia, M. J., & Kennedy, P. J. (1999). Evaluation of Typing Key Layouts for Stylus Input. Proceedings of The Human Factors and Ergonomics Society 43rd Annual Meeting.
[14]
Lewis, J. R., Potosnak, K. M., & Magyar, R. L. (1997). Keys and Keyboards. In M. G. Helander, T. K. Landauer & P. V. Prabhu (Eds.), Handbook of humancomputer interaction (2nd ed., pp. 1285--1315). Amsterdam: Elsevier Science.
[15]
MacKenzie, I. S. and Soukoreff, R. W. (2002). A Model of Two--Thumb Text Entry. In Proceedings of Graphics Interface '02, pp. 117--124.
[16]
MacKenzie, I. S. and Soukoreff, R. W. (2002). A chracter-level error analysis technique for evaluating text entry methods. In Proceedings of the 2nd Nordic Conference on Human-Computer Interaction (NordiCHI). Arhus, Denmark (Oct. 19--23). 243--24.
[17]
MacKenzie, I. S., and Soukoreff, R. W. (2003). Phrase sets for evaluating text entry techniques. Proc. of CHI EA \'1803, pp. 754--755. New York: ACM.
[18]
MacKenzie, I. S., and Tanaka-Ishii, K. (Eds.). (2007). Text Entry Systems: Mobility, Accessibility, Universality: Morgan Kaufmann Publishers.
[19]
MacKenzie, I. S., and Zhang, S. X. (1999) The design and evaluation of a high-performance soft keyboard. Proceedings of the ACM Conference on Human Factors in Computing Systems CHI '99, pp. 25--31. New York: ACM.
[20]
The New York Times, iPhone Keyboard Secrets. http://pogue.blogs.nytimes.com/2007/06/27/iphonekeyboard-secrets/
[21]
Rick, J. (2010). Performance optimizations of virtual keyboards for stroke-based text entry on a touch-based tabletop. ACM UIST, 77--86.
[22]
Rudchenko, D., Paek, T., and Badger, E. (2011). Text Text Revolution: A game that improves text entry on mobile touchscreen keyboards. Proc. Pervasive \'182011.
[23]
Rumelhart, D. E. and Norman, D. A. (1982), Simulating a Skilled Typist: A Study of Skilled CognitiveMotor Performance. Cognitive Science, 6: 1--36.
[24]
Wobbrock, J.O. (2007). Measures of text entry performance. Chapter 3 in I.S. MacKenzie and K. TanakaIshii (eds.), Text Entry Systems: Mobility, Accessibility, Universality. San Francisco: Morgan Kaufmann, pp. 47--74.
[25]
Wobbrock, J.O. and Myers, B.A. (2006). Analyzing the input stream for character-level errors in unconstrained text entry evaluations. ACM Transactions on Computer-Human Interaction 13 (4), pp. 458--489.
[26]
Zhai, S., Hunter, M., & Smith, B. A. (2002). Performance optimization of virtual keyboards. Human-Computer Interaction, 17(2,3), 89--129.
[27]
Zhai, S. and Kristensson, P.O. 2003. Shorthand writing on stylus keyboard. Proc. of CHI \'1803. ACM Press: 97104.
[28]
Zhai, S. and Kristensson, P.O. Introduction to Shape Writing (2006) IBM Research Report RJ10393 (A0611-006), November.
[29]
Zhai, S. and Kristensson, P.O. (2012). The wordgesture keyboard: reimagining keyboard interaction. Commun. ACM 55, 9 (September 2012), 91--101.

Cited By

View all
  • (2024)Accessible Gesture Typing on Smartphones for People with Low VisionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676447(1-11)Online publication date: 13-Oct-2024
  • (2023)GlanceWriter: Writing Text by Glancing Over Letters with GazeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581269(1-13)Online publication date: 19-Apr-2023
  • (2023)C-PAK: Correcting and Completing Variable-Length Prefix-Based Abbreviated KeystrokesACM Transactions on Computer-Human Interaction10.1145/354410130:1(1-35)Online publication date: 7-Mar-2023
  • Show More Cited By

Index Terms

  1. Octopus: evaluating touchscreen keyboard correction and recognition algorithms via

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
    April 2013
    3550 pages
    ISBN:9781450318990
    DOI:10.1145/2470654
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 27 April 2013

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. simulation
    2. text entry
    3. touch screen interaction

    Qualifiers

    • Research-article

    Conference

    CHI '13
    Sponsor:

    Acceptance Rates

    CHI '13 Paper Acceptance Rate 392 of 1,963 submissions, 20%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)20
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Accessible Gesture Typing on Smartphones for People with Low VisionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676447(1-11)Online publication date: 13-Oct-2024
    • (2023)GlanceWriter: Writing Text by Glancing Over Letters with GazeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581269(1-13)Online publication date: 19-Apr-2023
    • (2023)C-PAK: Correcting and Completing Variable-Length Prefix-Based Abbreviated KeystrokesACM Transactions on Computer-Human Interaction10.1145/354410130:1(1-35)Online publication date: 7-Mar-2023
    • (2022)Phrase-Gesture Typing on SmartphonesProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545683(1-11)Online publication date: 29-Oct-2022
    • (2021)iText: Hands-free Text Entry on an Imaginary Keyboard for Augmented Reality SystemsThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474788(815-825)Online publication date: 10-Oct-2021
    • (2021)Typing Efficiency and Suggestion Accuracy Influence the Benefits and Adoption of Word SuggestionsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445725(1-13)Online publication date: 6-May-2021
    • (2021)PhraseFlow: Designs and Empirical Studies of Phrase-Level InputProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445166(1-13)Online publication date: 6-May-2021
    • (2021)BackSwipe: Back-of-device Word-Gesture Interaction on SmartphonesProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445081(1-12)Online publication date: 6-May-2021
    • (2020)ThaiFlickProceedings of the 11th International Conference on Advances in Information Technology10.1145/3406601.3406615(1-8)Online publication date: 1-Jul-2020
    • (2020)Modeling Two Dimensional Touch PointingProceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology10.1145/3379337.3415871(858-868)Online publication date: 20-Oct-2020
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media