skip to main content
10.1145/3025453.3026027acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Float: One-Handed and Touch-Free Target Selection on Smartwatches

Published:02 May 2017Publication History

ABSTRACT

Touch interaction on smartwatches suffers from the awkwardness of having to use two hands and the "fat finger" problem. We present Float, a wrist-to-finger input approach that enables one-handed and touch-free target selection on smartwatches with high efficiency and precision using only commercially-available built-in sensors. With Float, a user tilts the wrist to point and performs an in-air finger tap to click. To realize Float, we first explore the appropriate motion space for wrist tilt and determine the clicking action (finger tap) through a user-elicitation study. We combine the photoplethysmogram (PPG) signal with accelerometer and gyroscope to detect finger taps with a recall of 97.9% and a false discovery rate of 0.4%. Experiments show that using just one hand, Float allows users to acquire targets with size ranging from 2mm to 10mm in less than 2s to 1s, meanwhile achieve much higher accuracy than direct touch in both stationary (>98.9%) and walking (>71.5%) contexts.

Skip Supplemental Material Section

Supplemental Material

pn4710-file3.mp4

mp4

46.5 MB

pn4710p.mp4

mp4

3 MB

p692-sun.mp4

mp4

245.2 MB

References

  1. Pär-Anders Albinsson and Shumin Zhai. 2003. High precision touch screen interaction. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '03), 105--112. DOI=http://dx.doi.org/10.1145/642611.642631 Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Shaikh Shawon Arefin Shimon, Courtney Lutton, Zichun Xu, Sarah Morrison-Smith, Christina Boucher and Jaime Ruiz. 2016. Exploring Non-touchscreen Gestures for Smartwatches. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 3822--3833. DOI=http://dx.doi.org/10.1145/2858036.2858385 Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. Daniel Ashbrook, Kent Lyons and Thad Starner. 2008. An investigation into round touchscreen wristwatch interaction. In Proceedings of the international conference on Human computer interaction with mobile devices and services (MobileHCI '08), 311--314. DOI=http://dx.doi.org/10.1145/1409240.1409276 Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. Patrick Baudisch and Gerry Chu. 2009. Back-of-device interaction allows creating very small touch devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 1923--1932. DOI=http://dx.doi.org/10.1145/1518701.1518995 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Hrvoje Benko, Andrew D. Wilson and Patrick Baudisch. 2006. Precise selection techniques for multitouch screens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 1263--1272. DOI=http://dx.doi.org/10.1145/1124772.1124963. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Xiaojun Bi, Yang Li and Shumin Zhai. 2013. FFitts law: modeling finger touch with fitts' law. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 1363--1372. DOI=http://dx.doi.org/10.1145/2470654.2466180 Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Alex Butler, Shahram Izadi and Steve Hodges. 2008. SideSight: multi-"touch" interaction around small devices. In Proceedings of the 21st annual ACM symposium on User interface software and technology (UIST '08), 201--204. DOI=http://dx.doi.org/10.1145/1449715.1449746 Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Edwin Chan, Teddy Seyed, Wolfgang Stuerzlinger, Xing-Dong Yang and Frank Maurer. 2016. User Elicitation on Single-hand Microgestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 3403--3414. DOI=http://dx.doi.org/10.1145/2858036.2858589 Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Youli Chang, Sehi L'Yi, Kyle Koh and Jinwook Seo. 2015. Understanding Users' Touch Behavior on Large Mobile Touch-Screens and Assisted Targeting by Tilting Gesture. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15), 1499--1508. DOI=http://dx.doi.org/10.1145/2702123.2702425 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. Andrew Crossan and Roderick Murray-Smith. 2004. Variability in wrist-tilt accelerometer based gesture interfaces. In Proceedings of the International Conference on Mobile Human-Computer Interaction (MobileHCI '04), 144--155. Google ScholarGoogle ScholarCross RefCross Ref
  11. Andrew Crossan, John Williamson, Stephen Brewster and Rod Murray-Smith. 2008. Wrist rotation for interaction in mobile contexts. In Proceedings of the international conference on Human computer interaction with mobile devices and services (MobileHCI '08), 435--438. DOI=http://dx.doi.org/10.1145/1409240.1409307 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Artem Dementyev and Joseph A. Paradiso. 2014. WristFlex: low-power gesture input with wrist-worn pressure sensors. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14), 161--166. DOI=http://doi.acm.org/10.1145/2642918.2647396Google ScholarGoogle Scholar
  13. Hui Ding, Goce Trajcevski, Peter Scheuermann, Xiaoyue Wang and Eamonn Keogh. 2008. Querying and mining of time series data: experimental comparison of representations and distance measures. In Proceedings of the VLDB Endowment 1, 2: 1542--1552. DOI=http://dx.doi.org/10.14778/1454159.1454226 Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Anhong Guo and Tim Paek. 2016. Exploring tilt for no-touch, wrist-only interactions on smartwatches. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '16), 17--28. DOI=http://dx.doi.org/10.1145/2935334.2935345 Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Chris Harrison and Scott E. Hudson. 2009. Abracadabra: wireless, high-precision, and unpowered finger input for very small mobile devices. In Proceedings of the 22nd annual ACM symposium on User interface software and technology (UIST '09), 121--124. DOI=http://dx.doi.org/10.1145/1622176.1622199 Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Da-Yuan Huang, Liwei Chan, Shuo Yang, Fan Wang, Rong-Hao Liang, De-Nian Yang, Yi-Ping Hung, and Bing-Yu Chen. 2016. DigitSpace: Designing Thumbto-Fingers Touch Interfaces for One-Handed and EyesFree Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 1526--1537. DOI=http://dx.doi.org/10.1145/2858036.2858483 Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Eleanor Jones, Jason Alexander, Andreas Andreou, Pourang Irani and Sriram Subramanian. 2010. GesText: accelerometer-based gestural text-entry systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10), 2173--2182. DOI=http://dx.doi.org/10.1145/1753326.1753655 Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Jun Gong, Xing-Dong Yang, and Pourang Irani. 2016. WristWhirl: One-handed Continuous Smartwatch Input using Wrist Gestures. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST '16), 861--872. DOI=https://doi.org/10.1145/2984511.2984563Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Jungsoo Kim, Jiasheng He, Kent Lyons and Thad Starner. 2007. The gesture watch: A wireless contactfree gesture based wrist interface. In Proceedings of the International Symposium on Wearable Computers (ISWC '07), 1--8. DOI=http://dx.doi.org/10.1109/ISWC.2007.4373770 Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Ju-Whan Kim, Han-Jong Kim and Tek-Jin Nam. 2016. M. Gesture: An Acceleration-Based Gesture Authoring System on Multiple Handheld and Wearable Devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 2307--2318. DOI=http://dx.doi.org/10.1145/2858036.2858358 Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Benjamin Lafreniere, Carl Gutwin, Andy Cockburn and Tovi Grossman. 2016. Faster Command Selection on Touchscreen Watches. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 4663--4674. DOI=http://dx.doi.org/10.1145/2858036.2858166 Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Loclair C, Gustafson S, Baudisch P. 2010. PinchWatch: a wearable device for one-handed microinteractions. In Proceedings of the International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '10)Google ScholarGoogle Scholar
  23. Gierad Laput, Robert Xiao, Xiang Chen, Scott E. Hudson and Chris Harrison. 2014. Skin buttons: cheap, small, low-powered and clickable fixed-icon laser projectors. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14), 389--394. DOI=http://dx.doi.org/10.1145/2642918.2647356 Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. I. Scott MacKenzie and Robert J. Teather. 2012. FittsTilt: the application of Fitts' law to tilt-based interaction. In Proceedings of the Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12), 568--577. DOI=http://dx.doi.org/10.1145/2399016.2399103 Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Florian Müller, Sebastian Günther, Niloofar Dezfuli, Mohammadreza Khalilbeigi and Max Mühlhäuser. 2016. ProxiWatch: Enhancing Smartwatch Interaction through Proximity-based Hand Input. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16), 2617--2624. DOI=http://dx.doi.org/10.1145/2851581.2892450 Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Ian Oakley and Sile O'Modhrain. 2005. Tilt to scroll: evaluating a motion based vibrotactile mobile interface. In First Joint Eurohaptics Conference and Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. World Haptics Conference (HAPTICS '05), 40--49. DOI=http://dx.doi.org/10.1109/WHC.2005.138 Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Ian Oakley and Doyoung Lee. 2014. Interaction on the edge: offset sensing for small devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 169--178. DOI=http://dx.doi.org/10.1145/2556288.2557138 Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. Ian Oakley, Doyoung Lee, M.D Rasel Islam and Augusto Esteves. 2015. Beats: Tapping Gestures for Smart Watches. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15), 1237--1246. DOI=http://dx.doi.org/10.1145/2702123.2702226 Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Masa Ogata and Michita Imai. 2015. SkinWatch: skin gesture interaction for smart watch. In Proceedings of the Augmented Human International Conference (AH '15), 21--24. DOI=http://dx.doi.org/10.1145/2735711.2735830 Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Stephen Oney, Chris Harrison, Amy Ogan and Jason Wiese. 2013. ZoomBoard: a diminutive qwerty soft keyboard using iterative zooming for ultra-small devices. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 2799--2802. DOI=http://dx.doi.org/10.1145/2470654.2481387 Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. Anna Ostberg and Nada Matic. 2015. Hover cursor: Improving touchscreen acquisition of small targets with hover-enabled pre-selection. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15), 17231728. DOI=http://dx.doi.org/10.1145/2702613.2732903 Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. Kurt Partridge, Saurav Chatterjee, Vibha Sazawal, Gaetano Borriello and Roy Want. 2002. TiltType: accelerometer-supported text entry for very small devices. In Proceedings of the 15th annual ACM symposium on User interface software and technology (UIST '02), 201--204. DOI=http://dx.doi.org/10.1145/571985.572013 Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Richard L. Potter, Linda J. Weldon and Ben Shneiderman. 1988. Improving the accuracy of touch screens: an experimental evaluation of three strategies. In Proceedings of the SIGCHI conference on Human factors in computing systems (CHI '88), 27--32. DOI=http://dx.doi.org/10.1145/57167.57171 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. Jun Rekimoto. 1996. Tilting operations for small screen interfaces. In Proceedings of the 9th annual ACM symposium on User interface software and technology (UIST '96), 167--168. DOI=http://dx.doi.org/10.1145/237091.237115 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Jun Rekimoto. 2001. Gesturewrist and gesturepad: Unobtrusive wearable interaction devices. In Proceedings of the International Symposium on Wearable Computers (ISWC '01), 21--27. Google ScholarGoogle ScholarCross RefCross Ref
  36. Anne Roudaut, Stéphane Huot and Eric Lecolinet. 2008. TapTap and MagStick: improving one-handed target acquisition on small touch-screens. In Proceedings of the working conference on Advanced visual interfaces (AVI '08), 146--153. DOI=http://dx.doi.org/10.1145/1385569.1385594 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. Keigo Shima, Kazusa Onishi, Ryosuke Takada, Takuya Adachi, Buntarou Shizuki and Jiro Tanaka. 2016. Investigating Accuracy of Tilting Operation on Wristworn Devices with Touchscreens. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 2705--2711. DOI=http://dx.doi.org/10.1145/2851581.2892377 Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. Robert J. Teather and I. Scott MacKenzie. 2014. Position vs. velocity control for tilt-based interaction. In Proceedings of Graphics Interface (GI '14), 51--58.Google ScholarGoogle Scholar
  39. Daniel Vogel, and Patrick Baudisch. 2007. Shift: a technique for operating pen-based interfaces using touch. In Proceedings of the SIGCHI Conference on Human Factors in Computing System (CHI '07), 657666. DOI=http://dx.doi.org/10.1145/1240624.1240727Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Lars Weberg, Torbjörn Brange and Åsa Wendelbo Hansson. 2001. A piece of butter on the PDA display. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '01), 435--436. DOI=http://dx.doi.org/10.1145/1240624.1240727 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Hongyi Wen, Julian Ramos Rojas and Anind K. Dey. 2016. Serendipity: Finger Gesture Recognition using an Off-the-Shelf Smartwatch. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 3847--3851. DOI=http://dx.doi.org/10.1145/2858036.2858466 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Daniel Wigdor and Ravin Balakrishnan. 2003. TiltText: using tilt for text input to mobile phones. In Proceedings of the 16th annual ACM symposium on User interface software and technology (UIST '03), 8190. DOI=http://dx.doi.org/10.1145/964696.964705 Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Daniel Wigdor, Clifton Forlines, Patrick Baudisch, John Barnwell and Chia Shen. 2007. Lucid touch: a see-through mobile device. In Proceedings of the 20th annual ACM symposium on User interface software and technology (UIST '07), 269--278. DOI=http://dx.doi.org/10.1145/1294211.1294259 Google ScholarGoogle ScholarDigital LibraryDigital Library
  44. Jacob O. Wobbrock, Meredith Ringel Morris and Andrew D. Wilson. 2009. User-defined gestures for surface computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 1083--1092. DOI=http://dx.doi.org/10.1145/1518701.1518866 Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Haijun Xia, Tovi Grossman and George Fitzmaurice. 2015. NanoStylus: Enhancing Input on Ultra-Small Displays with a Finger-Mounted Stylus. In Proceedings of the 28th Annual ACM Symposium on User Interface Software and Technology (UIST '15), 447--456. DOI=http://dx.doi.org/10.1145/2807442.2807500 Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Robert Xiao, Gierad Laput and Chris Harrison. 2014. Expanding the input expressivity of smartwatches with mechanical pan, twist, tilt and click. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14) 193--196. DOI=http://dx.doi.org/10.1145/2556288.2557017 Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Chun Yu, Hongyi Wen, Wei Xiong, Xiaojun Bi and Yuanchun Shi. 2016. Investigating Effects of PostSelection Feedback for Acquiring Ultra-Small Targets on Touchscreen. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 4699--4710. DOI=http://dx.doi.org/10.1145/2858036.2858593 Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Yang Zhang, Junhan Zhou, Gierad Laput and Chris Harrison. 2016. SkinTrack: Using the Body as an Electrical Waveguide for Continuous Finger Tracking on the Skin. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16), 1491--1503. DOI=http://dx.doi.org/10.1145/2858036.2858082 Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. https://en.wikipedia.org/wiki/Butterworth_filterGoogle ScholarGoogle Scholar
  50. https://en.wikipedia.org/wiki/Pearson_productmoment_correlation_coefficientGoogle ScholarGoogle Scholar
  51. https://en.wikipedia.org/wiki/Preferred_walking_speedGoogle ScholarGoogle Scholar
  52. www.pulsesensor.comGoogle ScholarGoogle Scholar

Index Terms

  1. Float: One-Handed and Touch-Free Target Selection on Smartwatches

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
      May 2017
      7138 pages
      ISBN:9781450346559
      DOI:10.1145/3025453

      Copyright © 2017 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 2 May 2017

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      CHI '17 Paper Acceptance Rate600of2,400submissions,25%Overall Acceptance Rate6,199of26,314submissions,24%

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader