ABSTRACT
Touch interaction on smartwatches suffers from the awkwardness of having to use two hands and the "fat finger" problem. We present Float, a wrist-to-finger input approach that enables one-handed and touch-free target selection on smartwatches with high efficiency and precision using only commercially-available built-in sensors. With Float, a user tilts the wrist to point and performs an in-air finger tap to click. To realize Float, we first explore the appropriate motion space for wrist tilt and determine the clicking action (finger tap) through a user-elicitation study. We combine the photoplethysmogram (PPG) signal with accelerometer and gyroscope to detect finger taps with a recall of 97.9% and a false discovery rate of 0.4%. Experiments show that using just one hand, Float allows users to acquire targets with size ranging from 2mm to 10mm in less than 2s to 1s, meanwhile achieve much higher accuracy than direct touch in both stationary (>98.9%) and walking (>71.5%) contexts.
Supplemental Material
- Pär-Anders Albinsson and Shumin Zhai. 2003. High precision touch screen interaction. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '03), 105--112. DOI=http://dx.doi.org/10.1145/642611.642631 Google ScholarDigital Library
- Shaikh Shawon Arefin Shimon, Courtney Lutton, Zichun Xu, Sarah Morrison-Smith, Christina Boucher and Jaime Ruiz. 2016. Exploring Non-touchscreen Gestures for Smartwatches. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 3822--3833. DOI=http://dx.doi.org/10.1145/2858036.2858385 Google ScholarDigital Library
- Daniel Ashbrook, Kent Lyons and Thad Starner. 2008. An investigation into round touchscreen wristwatch interaction. In Proceedings of the international conference on Human computer interaction with mobile devices and services (MobileHCI '08), 311--314. DOI=http://dx.doi.org/10.1145/1409240.1409276 Google ScholarDigital Library
- Patrick Baudisch and Gerry Chu. 2009. Back-of-device interaction allows creating very small touch devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 1923--1932. DOI=http://dx.doi.org/10.1145/1518701.1518995 Google ScholarDigital Library
- Hrvoje Benko, Andrew D. Wilson and Patrick Baudisch. 2006. Precise selection techniques for multitouch screens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 1263--1272. DOI=http://dx.doi.org/10.1145/1124772.1124963. Google ScholarDigital Library
- Xiaojun Bi, Yang Li and Shumin Zhai. 2013. FFitts law: modeling finger touch with fitts' law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 1363--1372. DOI=http://dx.doi.org/10.1145/2470654.2466180 Google ScholarDigital Library
- Alex Butler, Shahram Izadi and Steve Hodges. 2008. SideSight: multi-"touch" interaction around small devices. In Proceedings of the 21st annual ACM symposium on User interface software and technology (UIST '08), 201--204. DOI=http://dx.doi.org/10.1145/1449715.1449746 Google ScholarDigital Library
- Edwin Chan, Teddy Seyed, Wolfgang Stuerzlinger, Xing-Dong Yang and Frank Maurer. 2016. User Elicitation on Single-hand Microgestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 3403--3414. DOI=http://dx.doi.org/10.1145/2858036.2858589 Google ScholarDigital Library
- Youli Chang, Sehi L'Yi, Kyle Koh and Jinwook Seo. 2015. Understanding Users' Touch Behavior on Large Mobile Touch-Screens and Assisted Targeting by Tilting Gesture. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15), 1499--1508. DOI=http://dx.doi.org/10.1145/2702123.2702425 Google ScholarDigital Library
- Andrew Crossan and Roderick Murray-Smith. 2004. Variability in wrist-tilt accelerometer based gesture interfaces. In Proceedings of the International Conference on Mobile Human-Computer Interaction (MobileHCI '04), 144--155. Google ScholarCross Ref
- Andrew Crossan, John Williamson, Stephen Brewster and Rod Murray-Smith. 2008. Wrist rotation for interaction in mobile contexts. In Proceedings of the international conference on Human computer interaction with mobile devices and services (MobileHCI '08), 435--438. DOI=http://dx.doi.org/10.1145/1409240.1409307 Google ScholarDigital Library
- Artem Dementyev and Joseph A. Paradiso. 2014. WristFlex: low-power gesture input with wrist-worn pressure sensors. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14), 161--166. DOI=http://doi.acm.org/10.1145/2642918.2647396Google Scholar
- Hui Ding, Goce Trajcevski, Peter Scheuermann, Xiaoyue Wang and Eamonn Keogh. 2008. Querying and mining of time series data: experimental comparison of representations and distance measures. In Proceedings of the VLDB Endowment 1, 2: 1542--1552. DOI=http://dx.doi.org/10.14778/1454159.1454226 Google ScholarDigital Library
- Anhong Guo and Tim Paek. 2016. Exploring tilt for no-touch, wrist-only interactions on smartwatches. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16), 17--28. DOI=http://dx.doi.org/10.1145/2935334.2935345 Google ScholarDigital Library
- Chris Harrison and Scott E. Hudson. 2009. Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (UIST '09), 121--124. DOI=http://dx.doi.org/10.1145/1622176.1622199 Google ScholarDigital Library
- Da-Yuan Huang, Liwei Chan, Shuo Yang, Fan Wang, Rong-Hao Liang, De-Nian Yang, Yi-Ping Hung, and Bing-Yu Chen. 2016. DigitSpace: Designing Thumbto-Fingers Touch Interfaces for One-Handed and EyesFree Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 1526--1537. DOI=http://dx.doi.org/10.1145/2858036.2858483 Google ScholarDigital Library
- Eleanor Jones, Jason Alexander, Andreas Andreou, Pourang Irani and Sriram Subramanian. 2010. GesText: accelerometer-based gestural text-entry systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 2173--2182. DOI=http://dx.doi.org/10.1145/1753326.1753655 Google ScholarDigital Library
- Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-handed Continuous Smartwatch Input using Wrist Gestures. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16), 861--872. DOI=https://doi.org/10.1145/2984511.2984563Google ScholarDigital Library
- Jungsoo Kim, Jiasheng He, Kent Lyons and Thad Starner. 2007. The gesture watch: A wireless contactfree gesture based wrist interface. In Proceedings of the International Symposium on Wearable Computers (ISWC '07), 1--8. DOI=http://dx.doi.org/10.1109/ISWC.2007.4373770 Google ScholarDigital Library
- Ju-Whan Kim, Han-Jong Kim and Tek-Jin Nam. 2016. M. Gesture: An Acceleration-Based Gesture Authoring System on Multiple Handheld and Wearable Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 2307--2318. DOI=http://dx.doi.org/10.1145/2858036.2858358 Google ScholarDigital Library
- Benjamin Lafreniere, Carl Gutwin, Andy Cockburn and Tovi Grossman. 2016. Faster Command Selection on Touchscreen Watches. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 4663--4674. DOI=http://dx.doi.org/10.1145/2858036.2858166 Google ScholarDigital Library
- Loclair C, Gustafson S, Baudisch P. 2010. PinchWatch: a wearable device for one-handed microinteractions. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '10)Google Scholar
- Gierad Laput, Robert Xiao, Xiang Chen, Scott E. Hudson and Chris Harrison. 2014. Skin buttons: cheap, small, low-powered and clickable fixed-icon laser projectors. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14), 389--394. DOI=http://dx.doi.org/10.1145/2642918.2647356 Google ScholarDigital Library
- I. Scott MacKenzie and Robert J. Teather. 2012. FittsTilt: the application of Fitts' law to tilt-based interaction. In Proceedings of the Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12), 568--577. DOI=http://dx.doi.org/10.1145/2399016.2399103 Google ScholarDigital Library
- Florian Müller, Sebastian Günther, Niloofar Dezfuli, Mohammadreza Khalilbeigi and Max Mühlhäuser. 2016. ProxiWatch: Enhancing Smartwatch Interaction through Proximity-based Hand Input. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 2617--2624. DOI=http://dx.doi.org/10.1145/2851581.2892450 Google ScholarDigital Library
- Ian Oakley and Sile O'Modhrain. 2005. Tilt to scroll: evaluating a motion based vibrotactile mobile interface. In First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference (HAPTICS '05), 40--49. DOI=http://dx.doi.org/10.1109/WHC.2005.138 Google ScholarDigital Library
- Ian Oakley and Doyoung Lee. 2014. Interaction on the edge: offset sensing for small devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 169--178. DOI=http://dx.doi.org/10.1145/2556288.2557138 Google ScholarDigital Library
- Ian Oakley, Doyoung Lee, M.D Rasel Islam and Augusto Esteves. 2015. Beats: Tapping Gestures for Smart Watches. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15), 1237--1246. DOI=http://dx.doi.org/10.1145/2702123.2702226 Google ScholarDigital Library
- Masa Ogata and Michita Imai. 2015. SkinWatch: skin gesture interaction for smart watch. In Proceedings of the Augmented Human International Conference (AH '15), 21--24. DOI=http://dx.doi.org/10.1145/2735711.2735830 Google ScholarDigital Library
- Stephen Oney, Chris Harrison, Amy Ogan and Jason Wiese. 2013. ZoomBoard: a diminutive qwerty soft keyboard using iterative zooming for ultra-small devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 2799--2802. DOI=http://dx.doi.org/10.1145/2470654.2481387 Google ScholarDigital Library
- Anna Ostberg and Nada Matic. 2015. Hover cursor: Improving touchscreen acquisition of small targets with hover-enabled pre-selection. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15), 17231728. DOI=http://dx.doi.org/10.1145/2702613.2732903 Google ScholarDigital Library
- Kurt Partridge, Saurav Chatterjee, Vibha Sazawal, Gaetano Borriello and Roy Want. 2002. TiltType: accelerometer-supported text entry for very small devices. In Proceedings of the 15th annual ACM symposium on User interface software and technology (UIST '02), 201--204. DOI=http://dx.doi.org/10.1145/571985.572013 Google ScholarDigital Library
- Richard L. Potter, Linda J. Weldon and Ben Shneiderman. 1988. Improving the accuracy of touch screens: an experimental evaluation of three strategies. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '88), 27--32. DOI=http://dx.doi.org/10.1145/57167.57171 Google ScholarDigital Library
- Jun Rekimoto. 1996. Tilting operations for small screen interfaces. In Proceedings of the 9th annual ACM symposium on User interface software and technology (UIST '96), 167--168. DOI=http://dx.doi.org/10.1145/237091.237115 Google ScholarDigital Library
- Jun Rekimoto. 2001. Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In Proceedings of the International Symposium on Wearable Computers (ISWC '01), 21--27. Google ScholarCross Ref
- Anne Roudaut, Stéphane Huot and Eric Lecolinet. 2008. TapTap and MagStick: improving one-handed target acquisition on small touch-screens. In Proceedings of the working conference on Advanced visual interfaces (AVI '08), 146--153. DOI=http://dx.doi.org/10.1145/1385569.1385594 Google ScholarDigital Library
- Keigo Shima, Kazusa Onishi, Ryosuke Takada, Takuya Adachi, Buntarou Shizuki and Jiro Tanaka. 2016. Investigating Accuracy of Tilting Operation on Wristworn Devices with Touchscreens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 2705--2711. DOI=http://dx.doi.org/10.1145/2851581.2892377 Google ScholarDigital Library
- Robert J. Teather and I. Scott MacKenzie. 2014. Position vs. velocity control for tilt-based interaction. In Proceedings of Graphics Interface (GI '14), 51--58.Google Scholar
- Daniel Vogel, and Patrick Baudisch. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing System (CHI '07), 657666. DOI=http://dx.doi.org/10.1145/1240624.1240727Google ScholarDigital Library
- Lars Weberg, Torbjörn Brange and Åsa Wendelbo Hansson. 2001. A piece of butter on the PDA display. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '01), 435--436. DOI=http://dx.doi.org/10.1145/1240624.1240727 Google ScholarDigital Library
- Hongyi Wen, Julian Ramos Rojas and Anind K. Dey. 2016. Serendipity: Finger Gesture Recognition using an Off-the-Shelf Smartwatch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 3847--3851. DOI=http://dx.doi.org/10.1145/2858036.2858466 Google ScholarDigital Library
- Daniel Wigdor and Ravin Balakrishnan. 2003. TiltText: using tilt for text input to mobile phones. In Proceedings of the 16th annual ACM symposium on User interface software and technology (UIST '03), 8190. DOI=http://dx.doi.org/10.1145/964696.964705 Google ScholarDigital Library
- Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell and Chia Shen. 2007. Lucid touch: a see-through mobile device. In Proceedings of the 20th annual ACM symposium on User interface software and technology (UIST '07), 269--278. DOI=http://dx.doi.org/10.1145/1294211.1294259 Google ScholarDigital Library
- Jacob O. Wobbrock, Meredith Ringel Morris and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 1083--1092. DOI=http://dx.doi.org/10.1145/1518701.1518866 Google ScholarDigital Library
- Haijun Xia, Tovi Grossman and George Fitzmaurice. 2015. NanoStylus: Enhancing Input on Ultra-Small Displays with a Finger-Mounted Stylus. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15), 447--456. DOI=http://dx.doi.org/10.1145/2807442.2807500 Google ScholarDigital Library
- Robert Xiao, Gierad Laput and Chris Harrison. 2014. Expanding the input expressivity of smartwatches with mechanical pan, twist, tilt and click. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14) 193--196. DOI=http://dx.doi.org/10.1145/2556288.2557017 Google ScholarDigital Library
- Chun Yu, Hongyi Wen, Wei Xiong, Xiaojun Bi and Yuanchun Shi. 2016. Investigating Effects of PostSelection Feedback for Acquiring Ultra-Small Targets on Touchscreen. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 4699--4710. DOI=http://dx.doi.org/10.1145/2858036.2858593 Google ScholarDigital Library
- Yang Zhang, Junhan Zhou, Gierad Laput and Chris Harrison. 2016. SkinTrack: Using the Body as an Electrical Waveguide for Continuous Finger Tracking on the Skin. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 1491--1503. DOI=http://dx.doi.org/10.1145/2858036.2858082 Google ScholarDigital Library
- https://en.wikipedia.org/wiki/Butterworth_filterGoogle Scholar
- https://en.wikipedia.org/wiki/Pearson_productmoment_correlation_coefficientGoogle Scholar
- https://en.wikipedia.org/wiki/Preferred_walking_speedGoogle Scholar
- www.pulsesensor.comGoogle Scholar
Index Terms
- Float: One-Handed and Touch-Free Target Selection on Smartwatches
Recommendations
Around-device finger input on commodity smartwatches with learning guidance through discoverability
AbstractUser interactions with smartwatches are limited due to one-finger usage on small touch screens. However, smartwatch interaction capabilities can be extended beyond the screen into the around-device space by leveraging multi-finger interactions. ...
Highlights- We present user-preferred finger gestures and postures to interact with a smartwatch.
- We developed a smartwatch prototype to detect finger activities around the device.
- Next, we explored two guidance techniques: Crib-Sheet and ...
MyoTilt: a target selection method for smartwatches using the tilting operation and electromyography
MobileHCI '18: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and ServicesWe present the MyoTilt target selection method for smartwatches, which employs a combination of a tilt operation and electromyography (EMG). First, a user tilts his/her arm to indicate the direction of cursor movement on the smartwatch; then s/he ...
WristWhirl: One-handed Continuous Smartwatch Input using Wrist Gestures
UIST '16: Proceedings of the 29th Annual Symposium on User Interface Software and TechnologyWe propose and study a new input modality, WristWhirl, that uses the wrist as an always-available joystick to perform one-handed continuous input on smartwatches. We explore the influence of the wrist's bio-mechanical properties for performing gestures ...
Comments