Shariff AM Faleel
Yishuo Liu
Bradley Rey
Da-Yuan Huang
Wei Li
ABSTRACT
Three state virtual keyboards which differentiate contact events between released, touched, and pressed states have the potential to improve overall typing experience and reduce the gap between virtual keyboards and physical keyboards. Incorporating force sensitivity, three-state virtual keyboards can utilize a force threshold to better classify a contact event. However, our limited knowledge of how force plays a role during typing on virtual keyboards limits further progress. Through a series of studies we observe that using a uniform threshold is not an optimal approach. Furthermore, the force being applied while typing varies significantly across the keys and among participants. As such, we propose three different approaches to further improve the uniform threshold. We show that a carefully selected non-uniform threshold function could be sufficient in delineating typing events on a three-state keyboard. Finally, we conclude our work with lessons learned, suggestion for future improvements, and comparisons with current methods available.
Footnotes
Supplemental Material
- Thomas Bekken Aschim, Julie Lidahl Gjerstad, Lars Vidar Lien, Rukaiya Tahsin, and Frode Eika Sandnes. 2019. Are Split Tablet Keyboards Better? A Study of Soft Keyboard Layout and Hand Posture. Lecture Notes in Computer Science(2019), 647–655. https://doi.org/10.1007/978-3-030-29387-1_37Google Scholar
Digital Library
- Myroslav Bachynskyi, Gregorio Palmas, Antti Oulasvirta, Jürgen Steimle, and Tino Weinkauf. 2015. Performance and Ergonomics of Touch Surfaces. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2702123.2702607Google ScholarDigital Library
- James V. Bradley. 1958. Complete Counterbalancing of Immediate Sequential Effects in a Latin Square Design. J. Amer. Statist. Assoc. 53, 282 (1958), 525–528. https://doi.org/10.1080/01621459.1958.10501456 arXiv:https://www.tandfonline.com/doi/pdf/10.1080/01621459.1958.10501456Google ScholarCross Ref
- Daewoong Choi, Hyeonjoong Cho, and Joono Cheong. 2015. Improving Virtual Keyboards When All Finger Positions Are Known. In Proceedings of the 28th Annual ACM Symposium on User Interface Software & Technology. https://doi.org/10.1145/2807442.2807491Google ScholarDigital Library
- Amrish O. Chourasia, Douglas A. Wiegmann, Karen B. Chen, Curtis B. Irwin, and Mary E. Sesto. 2013. Effect of Sitting or Standing on Touch Screen Performance and Touch Characteristics. Human Factors 55, 4 (2013), 789–802. https://doi.org/10.1177/0018720812470843 arXiv:https://doi.org/10.1177/0018720812470843PMID: 23964418.Google ScholarCross Ref
- Matthew J. C. Crump and Gordon D. Logan. 2010. Warning: This Keyboard Will Deconstruct- the Role of the Keyboard in Skilled Typewriting. Psychonomic Bulletin & Review 17, 3 (2010), 394–399. https://doi.org/10.3758/pbr.17.3.394Google ScholarCross Ref
- Vivek Dhakal, Anna Maria Feit, Per Ola Kristensson, and Antti Oulasvirta. 2018. Observations on Typing from 136 Million Keystrokes. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3173574.3174220Google ScholarDigital Library
- Jörg Edelmann, Philipp Mock, Andreas Schilling, Peter Gerjets, Wolfgang Rosenstiel, and Wolfgang Straßer. 2012. Towards the keyboard of oz. In Proceedings of the 2012 ACM international conference on Interactive tabletops and surfaces - ITS ’12. https://doi.org/10.1145/2396636.2396662Google ScholarDigital Library
- Anna Maria Feit, Daryl Weir, and Antti Oulasvirta. 2016. How We Type. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2858036.2858233Google ScholarDigital Library
- Tamara Fiedler and Yasemin Vardar. 2019. A Novel Texture Rendering Approach for Electrostatic Displays. In International Workshop on Haptic and Audio Interaction Design - HAID2019. Lille, France. https://hal.archives-ouvertes.fr/hal-02011782Google Scholar
- Leah Findlater and Jacob Wobbrock. 2012. Personalized input. In Proceedings of the 2012 ACM annual conference on Human Factors in Computing Systems - CHI ’12. https://doi.org/10.1145/2207676.2208520Google ScholarDigital Library
- Leah Findlater, Jacob O. Wobbrock, and Daniel Wigdor. 2011. Typing on flat glass. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI ’11. https://doi.org/10.1145/1978942.1979301Google ScholarDigital Library
- Mayank Goel, Leah Findlater, and Jacob Wobbrock. 2012. WalkType: Using Accelerometer Data to Accomodate Situational Impairments in Mobile Touch Screen Text Entry. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Austin, Texas, USA) (CHI ’12). Association for Computing Machinery, New York, NY, USA, 2687–2696. https://doi.org/10.1145/2207676.2208662Google ScholarDigital Library
- Yizheng Gu, Chun Yu, Xuanzhong Chen, ZHUOJUN LI, and Yuanchun Shi. 2021. TypeBoard: Identifying Unintentional Touch on Pressure-Sensitive Touchscreen Keyboards. In The 34th Annual ACM Symposium on User Interface Software and Technology. nil. https://doi.org/10.1145/3472749.3474770Google ScholarDigital Library
- Tatsuhito Hasegawa and Tatsuya Hatakenaka. 2019. Touch-Typing Detection Using Eyewear: Toward Realizing a New Interaction for Typing Applications. Sensors 19, 9 (2019). https://doi.org/10.3390/s19092022Google ScholarCross Ref
- Min-Chieh Hsiu, Da-Yuan Huang, Chi An Chen, Yu-Chih Lin, Yi ping Hung, De-Nian Yang, and Mike Chen. 2016. ForceBoard. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct. nil. https://doi.org/10.1145/2957265.2961827Google ScholarDigital Library
- Seokhee Jeon, Hongchae Lee, Jiyoung Jung, and Jin Ryong Kim. 2018. User-Adaptive Key Click Vibration on Virtual Keyboard. Mobile Information Systems 2018 (2018), 1–12. https://doi.org/10.1155/2018/6126140Google ScholarCross Ref
- Xinhui Jiang, Yang Li, Jussi P.P. Jokinen, Viet Ba Hirvola, Antti Oulasvirta, and Xiangshi Ren. 2020. How We Type: Eye and Finger Movement Strategies in Mobile Typing. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376711Google ScholarDigital Library
- Huhn Kim, Seungyoun Yi, and So-Yeon Yoon. 2019. Exploring touch feedback display of virtual keyboards for reduced eye movements. Displays 56(2019), 38–48. https://doi.org/10.1016/j.displa.2018.11.004Google ScholarCross Ref
- Jeong Ho Kim, Lovenoor Aulck, Michael C Bartha, Christy A Harper, and Peter W Johnson. 2012. Are There Differences in Force Exposures and Typing Productivity Between Touchscreen and Conventional Keyboard?Proceedings of the Human Factors and Ergonomics Society Annual Meeting 56, 1(2012), 1104–1108. https://doi.org/10.1177/1071181312561240Google ScholarCross Ref
- Jin Ryong Kim, Xiaowei Dai, Xiang Cao, Carl Picciotto, Desney Tan, and Hong Z. Tan. 2012. A Masking Study of Key-Click Feedback Signals on a Virtual Keyboard. Springer Berlin Heidelberg, 247–257. https://doi.org/10.1007/978-3-642-31401-8_23Google ScholarDigital Library
- Jin Ryong Kim and Hong Z. Tan. 2014. Haptic Feedback Intensity Affects Touch Typing Performance on a Flat Keyboard. Springer Berlin Heidelberg, 369–375. https://doi.org/10.1007/978-3-662-44193-0_46Google ScholarCross Ref
- Jin Ryong Kim and Hong Z. Tan. 2014. A study of touch typing performance with keyclick feedback. In 2014 IEEE Haptics Symposium (HAPTICS). https://doi.org/10.1109/haptics.2014.6775459Google ScholarCross Ref
- Jin Ryong Kim and Hong Z. Tan. 2015. Effect of information content in sensory feedback on typing performance using a flat keyboard. In 2015 IEEE World Haptics Conference (WHC). https://doi.org/10.1109/whc.2015.7177718Google ScholarCross Ref
- Sunjun Kim and Geehyuk Lee. 2012. Typing on a Touch Surface: Effect of Feedback with Horizontal Touch Keyboard and Vertical Display Setup. In The 10th Asia Pacific Conference on Computer Human Interaction. Human Centered Design Organization and ACM, 525–530.Google Scholar
- Sunjun Kim, Jeongmin Son, Geehyuk Lee, Hwan Kim, and Woohun Lee. 2013. TapBoard. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/2470654.2470733Google ScholarDigital Library
- Ue-Hwan Kim, Sahng-Min Yoo, and Jong-Hwan Kim. 2019. I-Keyboard: Fully Imaginary Keyboard on Touch Devices Empowered By Deep Neural Decoder. CoRR (2019). arxiv:1907.13285 [cs.HC] http://arxiv.org/abs/1907.13285v1Google Scholar
- Yuki Kuno and Buntarou Shizuki. 2017. Meyboard: A QWERTY-Based Soft Keyboard for Touch-Typing on Tablets. Springer International Publishing, 193–207. https://doi.org/10.1007/978-3-319-58077-7_16Google ScholarCross Ref
- Sunghyuk Kwon, Donghun Lee, and Min K. Chung. 2009. Effect of Key Size and Activation Area on the Performance of a Regional Error Correction Method in a Touch-Screen Qwerty Keyboard. International Journal of Industrial Ergonomics 39, 5(2009), 888–893. https://doi.org/10.1016/j.ergon.2009.02.013Google ScholarCross Ref
- Lik Hang Lee, Ngo Yan Yeung, Tristan Braud, Tong Li, Xiang Su, and Pan Hui. 2020. Force9: Force-assisted Miniature Keyboard on Smart Wearables. In Proceedings of the 2020 International Conference on Multimodal Interaction. nil. https://doi.org/10.1145/3382507.3418827Google ScholarDigital Library
- Blaine Lewis, Greg d’Eon, Andy Cockburn, and Daniel Vogel. 2020. KeyMap: Improving Keyboard Shortcut Vocabulary Using Norman’s Mapping. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–10. https://doi.org/10.1145/3313831.3376483Google ScholarDigital Library
- Changshuan Li. 2010. Coping strategies for fast delivery in simultaneous interpretation. The Journal of Specialised Translation 13 (2010), 19–25.Google Scholar
- Frank Chun Yat Li, Leah Findlater, and Khai N. Truong. 2013. Effects of Hand Drift While Typing on Touchscreens. In Proceedings of Graphics Interface 2013 (Regina, Sascatchewan, Canada) (GI ’13). Canadian Information Processing Society, CAN, 95–98.Google Scholar
- Frank Chun Yat Li, Richard T. Guy, Koji Yatani, and Khai N. Truong. 2011. The 1line Keyboard: A QWERTY Layout in a Single Line. In Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology (Santa Barbara, California, USA) (UIST ’11). Association for Computing Machinery, New York, NY, USA, 461–470. https://doi.org/10.1145/2047196.2047257Google ScholarDigital Library
- Ming-I Brandon Lin, Ruei-Hong Hong, and Yu-Ping Huang. 2020. Influence of Virtual Keyboard Design and Usage Posture on Typing Performance and Muscle Activity During Tablet Interaction. Ergonomics 63, 10 (2020), 1312–1328. https://doi.org/10.1080/00140139.2020.1778097Google ScholarCross Ref
- Ming-I Brandon Lin, Ruei-Hong Hong, and Yu-Ping Huang. 2020. Influence of virtual keyboard design and usage posture on typing performance and muscle activity during tablet interaction. Ergonomics 63, 10 (2020), 1312–1328. https://doi.org/10.1080/00140139.2020.1778097 arXiv:https://doi.org/10.1080/00140139.2020.1778097PMID: 32496886.Google ScholarCross Ref
- Yiqin Lu, Chun Yu, Xin Yi, Yuanchun Shi, and Shengdong Zhao. 2017. Blindtype. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 2 (2017), 1–24. https://doi.org/10.1145/3090083Google ScholarDigital Library
- Zhaoyuan Ma, Darren Edge, Leah Findlater, and Hong Z. Tan. 2015. Haptic keyclick feedback improves typing speed and reduces typing errors on a flat keyboard. In 2015 IEEE World Haptics Conference (WHC). https://doi.org/10.1109/whc.2015.7177717Google ScholarCross Ref
- Zhaoyuan Ma, Darren Edge, Leah Findlater, and Hong Z. Tan. 2015. Haptic keyclick feedback improves typing speed and reduces typing errors on a flat keyboard. In 2015 IEEE World Haptics Conference (WHC). 220–227. https://doi.org/10.1109/WHC.2015.7177717Google ScholarCross Ref
- I. Scott MacKenzie and R. William Soukoreff. 2002. Text Entry for Mobile Computing: Models and Methods, Theory and Practice. Human–Computer Interaction 17, 2-3 (2002), 147–198. https://doi.org/10.1080/07370024.2002.9667313 arXiv:https://www.tandfonline.com/doi/pdf/10.1080/07370024.2002.9667313Google ScholarCross Ref
- I. Scott MacKenzie and R. William Soukoreff. 2003. Phrase sets for evaluating text entry techniques. In CHI ’03 extended abstracts on Human factors in computing systems - CHI ’03. https://doi.org/10.1145/765891.765971Google ScholarDigital Library
- Dan Odell. 2015. On-screen keyboards. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services. https://doi.org/10.1145/2785830.2785848Google ScholarDigital Library
- Dan Odell. 2015. On-Screen Keyboards: Does the Presence of Feedback or Tactile Landmarks Improve Typing Performance?. In Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services (Copenhagen, Denmark) (MobileHCI ’15). Association for Computing Machinery, New York, NY, USA, 131–136. https://doi.org/10.1145/2785830.2785848Google ScholarDigital Library
- Dan Odell and Eric Faggin. 2014. The Typing Performance and Preference Costs of Reducing Tactile Feedback and Tactile Landmarks in Tablet Keyboards. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 58, 1(2014), 1790–1794. https://doi.org/10.1177/1541931214581373Google ScholarCross Ref
- Sarangi P. Parikh and Joel M. Esposito. 2012. Negative Feedback for Small Capacitive Touchscreen Interfaces: a Usability Study for Data Entry Tasks. IEEE Transactions on Haptics 5, 1 (2012), 39–47. https://doi.org/10.1109/toh.2011.71Google ScholarDigital Library
- Radosław Puka, Piotr Łebkowski, and Jerzy Duda. 2021. One Row Keyboard: The Concept of Designing a Common Layout for Physical and Virtual Keyboards. Electronics 10, 6 (2021). https://doi.org/10.3390/electronics10060663Google ScholarCross Ref
- Ely Rabin and Andrew M. Gordon. 2004. Tactile Feedback Contributes To Consistency of Finger Movements During Typing. Experimental Brain Research 155, 3 (2004), 362–369. https://doi.org/10.1007/s00221-003-1736-6Google ScholarCross Ref
- Gregory Reardon, Nikolas Kastor, Yitian Shao, and Yon Visell. 2020. Elastowave: Localized Tactile Feedback in a Soft Haptic Interface via Focused Elastic Waves. In 2020 IEEE Haptics Symposium (HAPTICS). https://doi.org/10.1109/haptics45997.2020.ras.hap20.25.aa4d97aaGoogle ScholarCross Ref
- Mark Richardson, Matt Durasoff, and Robert Wang. 2020. Decoding Surface Touch Typing from Hand-Tracking. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology. https://doi.org/10.1145/3379337.3415816Google ScholarDigital Library
- Dominik Schmidt, Florian Block, and Hans Gellersen. 2009. A Comparison of Direct and Indirect Multi-touch Input for Large Surfaces. Springer Berlin Heidelberg, 582–594. https://doi.org/10.1007/978-3-642-03655-2_65Google ScholarDigital Library
- Andrew Sears, Julie A. Jacko, Josey Chu, and Francisco Moro. 2001. The Role of Visual Search in the Design of Effective Soft Keyboards. Behaviour & Information Technology 20, 3 (2001), 159–166. https://doi.org/10.1080/01449290110049790Google ScholarCross Ref
- Weinan Shi, Chun Yu, Xin Yi, Zhen Li, and Yuanchun Shi. 2018. Toast. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 1 (2018), 1–23. https://doi.org/10.1145/3191765Google ScholarDigital Library
- Paul D. Varcholik, Joseph J. LaViola, and Charles E. Hughes. 2012. Establishing a Baseline for Text Entry for a Multi-Touch Virtual Keyboard. International Journal of Human-Computer Studies 70, 10 (2012), 657–672. https://doi.org/10.1016/j.ijhcs.2012.05.007Google ScholarDigital Library
- Daryl Weir, Henning Pohl, Simon Rogers, Keith Vertanen, and Per Ola Kristensson. 2014. Uncertain text entry on mobile devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. nil. https://doi.org/10.1145/2556288.2557412Google ScholarDigital Library
- Pui Chung Wong, Kening Zhu, Xing-Dong Yang, and Hongbo Fu. 2020. Exploring Eyes-Free Bezel-Initiated Swipe on Round Smartwatches. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376393Google ScholarDigital Library
- Zhican Yang, Chun Yu, Xin Yi, and Yuanchun Shi. 2019. Investigating Gesture Typing for Indirect Touch. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 3, 3 (2019), 1–22. https://doi.org/10.1145/3351275Google ScholarDigital Library
- Xin Yi, Chen Wang, Xiaojun Bi, and Yuanchun Shi. 2020. PalmBoard: Leveraging Implicit Touch Pressure in Statistical Decoding for Indirect Text Entry. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3313831.3376441Google ScholarDigital Library
- Xiao Ying Zhao, Ian A Spraggs, David A Pakula, and Tang Y Tan. 2021. 3d touch. US Patent App. 16/708,350.Google Scholar
- Mingyuan Zhong, Chun Yu, Qian Wang, Xuhai Xu, and Yuanchun Shi. 2018. ForceBoard. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. nil. https://doi.org/10.1145/3173574.3174102Google ScholarDigital Library
- Suwen Zhu, Jingjie Zheng, Shumin Zhai, and Xiaojun Bi. 2019. i’sFree. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. nil. https://doi.org/10.1145/3290605.3300678Google ScholarDigital Library
Index Terms
- T-Force: Exploring the Use of Typing Force for Three State Virtual Keyboards
Recommendations
Typing on Midair Virtual Keyboards: Exploring Visual Designs and Interaction Styles
Human-Computer Interaction – INTERACT 2021AbstractWe investigate typing on a QWERTY keyboard rendered in virtual reality. Our system tracks users’ hands in the virtual environment via a Leap Motion mounted on the front of a head mounted display. This allows typing on an auto-correcting midair ...
Force gestures: augmenting touch screen gestures with normal and tangential forces
UIST '11: Proceedings of the 24th annual ACM symposium on User interface software and technologyForce gestures are touch screen gestures augmented by the normal and tangential forces on the screen. In order to study the feasibility of the force gestures on a mobile touch screen, we implemented a prototype touch screen device that can sense the ...
Pen-based force display for precision manipulation in virtual environments
VRAIS '95: Proceedings of the Virtual Reality Annual International Symposium (VRAIS'95)We describe the structure of a force display recently implemented for precision manipulation of scaled or virtual environments. We discuss the advantages of direct-drive parallel manipulators over geared serial manipulators for human-robot interaction ...
Comments