Junjue Wang
ABSTRACT
A well-known bottleneck of contemporary mobile devices is the inefficient and error-prone touchscreen keyboard. In this paper, we propose UbiK, an alternative portable text-entry method that allows user to make keystrokes on conventional surfaces, e.g., wood desktop. UbiK enables text-input experience similar to that on a physical keyboard, but it only requires a keyboard outline printed on the surface or a piece of paper atop. The core idea is to leverage the microphone on a mobile device to accurately localize the keystrokes. To achieve fine-grained, centimeter scale granularity, UbiK extracts and optimizes the location-dependent multipath fading features from the audio signals, and takes advantage of the dual-microphone interface to improve signal diversity. We implement UbiK as an Android application. Our experiments demonstrate that UbiK is able to achieve above 95% of localization accuracy. Field trial involving first-time users shows that UbiK can significantly improve text-entry speed over current on-screen keyboards.
- M. Kölsch and M. Turk, "Keyboards without Keyboards: A Survey of Virtual Keyboards," in Proceedings of Sensing and Input for Media-centric Systems, 2002.Google Scholar
- K. Al Faraj, M. Mojahid, and N. Vigouroux, "BigKey: A Virtual Keyboard for Mobile Devices," in Proceedings of International Conference on Human-Computer Interaction, 2009. Google ScholarDigital Library
- M. Goel, A. Jansen, T. Mandel, S. N. Patel, and J. O. Wobbrock, "ContextType: Using Hand Posture Information to Improve Mobile Touch Screen Text Entry," in Proc. of ACM CHI, 2013. Google ScholarDigital Library
- O. Schoenleben and A. Oulasvirta, "Sandwich keyboard: Fast ten-finger typing on a mobile device with adaptive touch sensing on the back side," in Proc. of ACM MobileHCI, 2013. Google ScholarDigital Library
- S. Oney, C. Harrison, A. Ogan, and J. Wiese, "ZoomBoard: A Diminutive Qwerty Soft Keyboard Using Iterative Zooming for Ultra-small Devices," in Proc. of ACM CHI, 2013. Google ScholarDigital Library
- H. Du, T. Oggier, F. Lustenberger, and E. Charbon, "A Virtual Keyboard Based on True-3D Optical Ranging," in Proceedings of the British Machine Vision Conference, 2005.Google Scholar
- H. Roeber, J. Bacus, and C. Tomasi, "Typing in Thin Air: The Canesta Projection Keyboard - a New Method of Interaction with Electronic Devices," in ACM CHI Extended Abstracts, 2003. Google ScholarDigital Library
- C. Harrison, H. Benko, and A. D. Wilson, "OmniTouch: Wearable Multitouch Interaction Everywhere," in Proc. of ACM UIST, 2011. Google ScholarDigital Library
- Y. Chen, D. Lymberopoulos, J. Liu, and B. Priyantha, "FM-based Indoor Localization," in Proc. of ACM MobiSys, 2012. Google ScholarDigital Library
- T. Gustafsson, B. Rao, and M. Trivedi, "Source Localization in Reverberant Environments: Modeling and Statistical Analysis," IEEE Transactions on Speech and Audio Processing, vol. 11, no. 6, 2003.Google Scholar
- C. Peng, G. Shen, Y. Zhang, Y. Li, and K. Tan, "BeepBeep: a High Accuracy Acoustic Ranging System Using COTS Mobile Devices," in Proc. of ACM SenSys, 2007. Google ScholarDigital Library
- K. Ho and M. Sun, "Passive Source Localization Using Time Differences of Arrival and Gain Ratios of Arrival," IEEE Transactions on Signal Processing, vol. 56, no. 2, 2008. Google ScholarDigital Library
- S. Sen, J. Lee, K.-H. Kim, and P. Congdon, "Avoiding Multipath to Revive Inbuilding WiFi Localization," in Proc. of ACM MobiSys, 2013. Google ScholarDigital Library
- J. Wang and D. Katabi, "Dude, Where's My Card?: RFID Positioning That Works with Multipath and Non-Line of Sight," in Proc. of ACM SIGCOMM, 2013. Google ScholarDigital Library
- D. Tse and P. Viswanath, Fundamentals of Wireless Communication. Cambridge University Press, 2005. Google ScholarDigital Library
- M. Vorlander, Fundamentals of Acoustics, Modelling, Simulation, Algorithms and Acoustic Virtual Reality. Springer, 2008. Google ScholarDigital Library
- S. Sen, B. Radunovic, R. R. Choudhury, and T. Minka, "You Are Facing the Mona Lisa: Spot Localization Using PHY Layer Information," in Proc. of ACM MobiSys, 2012. Google ScholarDigital Library
- W. Cui, Z. Cao, and J. Wei, "Dual-Microphone Source Location Method in 2-D Space," in Proc. of IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2006.Google Scholar
- J. H. DiBiase, "A High-Accuracy, Low-Latency Technique for Talker Localization in Reverberant Environment," Ph.D. dissertation, Brown University, 2000.Google Scholar
- R. Blum, S. Kassam, and H. Poor, "Distributed Detection With Multiple Sensors I. Advanced topics," Proceedings of the IEEE, vol. 85, no. 1, 1997.Google Scholar
- P. Marquardt, A. Verma, H. Carter, and P. Traynor, "(Sp)iPhone: Decoding Vibrations from Nearby Keyboards Using Mobile Phone Accelerometers," in Proc. of ACM CCS, 2011. Google ScholarDigital Library
- T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. Springer, 2013.Google Scholar
- Smart Tool Co., "Sound Meter Pro." {Online}. Available: https://play.google.com/store/apps/details?id=kr.aboy.soundGoogle Scholar
- Monsoon Solutions, Inc., "Monsoon Power Monitor," http://www.msoon.com/LabEquipment/PowerMonitor/.Google Scholar
- I. S. MacKenzie and R. W. Soukoreff, "Phrase Sets for Evaluating Text Entry Techniques," in ACM CHI Extended Abstracts, 2003. Google ScholarDigital Library
- Swype Inc., "The Swype Virtual Keyboard," 2013. {Online}. Available: http://www.swype.com/Google Scholar
- S. Kim, J. Son, G. Lee, H. Kim, and W. Lee, "TapBoard: Making a Touch Screen Keyboard More Touchable," in Proc. of ACM CHI, 2013. Google ScholarDigital Library
- Z. Zhang, D. Chu, X. Chen, and T. Moscibroda, "SwordFight: Enabling a New Class of Phone-to-phone Action Games on Commodity Phones," in Proc. of ACM MobiSys, 2012. Google ScholarDigital Library
- D. Asonov and R. Agrawal, "Keyboard Acoustic Emanations," in IEEE Symposium on Security and Privacy, 2004.Google Scholar
- L. Zhuang, F. Zhou, and J. D. Tygar, "Keyboard Acoustic Emanations Revisited," ACM Transactions on Information System Security, vol. 13, no. 1, 2009. Google ScholarDigital Library
- J. Mantyjarvi, J. Koivumaki, and P. Vuori, "Keystroke Recognition for Virtual Keyboard," in Proc. of IEEE International Conference on Multimedia and Expo (ICME), 2002.Google Scholar
- C. Harrison, J. Schwarz, and S. E. Hudson, "TapSense: Enhancing Finger Interaction on Touch Surfaces," in Proc. of ACM UIST, 2011. Google ScholarDigital Library
- M. Ono, B. Shizuki, and J. Tanaka, "Touch & Activate: Adding Interactivity to Existing Objects Using Active Acoustic Sensing," in Proc. of ACM UIST, 2013. Google ScholarDigital Library
- J. A. Paradiso, C. K. Leo, N. Checka, and K. Hsiao, "Passive Acoustic Knock Tracking for Interactive Windows," in ACM CHI Extended Abstracts, 2002. Google ScholarDigital Library
- D. T. Pham, Z. Ji, M. Yang, Z. Wang, and M. Al-Kutubi, "A Novel Human-computer Interface Based on Passive Acoustic Localisation," in Proc. of International Conference on Human-computer Interaction, 2007. Google ScholarDigital Library
Index Terms
- Ubiquitous keyboard for small mobile devices: harnessing multipath fading for fine-grained keystroke localization
Recommendations
Demo: A paper keyboard for mobile devices
MobiSys '14: Proceedings of the 12th annual international conference on Mobile systems, applications, and servicesA well-known bottleneck of contemporary mobile devices is the inefficient and error-prone touchscreen keyboard. We have developed UbiK, an alternative portable text-entry method that allows user to type on a piece of paper, placed on solid surfaces like ...
Flickey: Flick-Based QWERTY Software Keyboard for Ultra-small Touch Screen Devices
Human-Computer Interaction. Interaction TechnologiesAbstractUltra-small touch screen devices (e.g., smartwatches) are required to be small and lightweight so that they can be worn on the body with no frustration. For this reason, users often have difficulties in selecting the correct keys, and thus ...
Text Entry for Ultra-Small Touchscreens Using a Fixed Cursor and Movable Keyboard
CHI EA '16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing SystemsEmerging ultra-small wearables like smartwatches pose a design challenge for touch-based text entry, due to the "fat finger" problem, wherein users struggle to select elements much smaller than their fingers. In order to address the challenge, we ...
Comments