skip to main content
10.1145/1166253.1166292acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
Article

Robust computer vision-based detection of pinching for one and two-handed gesture input

Published: 15 October 2006 Publication History

Abstract

We present a computer vision technique to detect when the user brings their thumb and forefinger together (a pinch gesture) for close-range and relatively controlled viewing circumstances. The technique avoids complex and fragile hand tracking algorithms by detecting the hole formed when the thumb and forefinger are touching; this hole is found by simple analysis of the connected components of the background segmented against the hand. Our Thumb and Fore-Finger Interface (TAFFI) demonstrates the technique for cursor control as well as map navigation using one and two-handed interactions.

Supplementary Material

JPG File (1166292.jpg)
index.html (index.html)
Slides from the presentation
ZIP File (p255-slides.zip)
Supplemental material for Robust computer vision-based detection of pinching for one and two-handed gesture input
Audio only (1166292.mp3)
Video (1166292.mp4)

References

[1]
Balakrishnan, R. and MacKenzie, I. S., Performance differences in the fingers, wrist, and forearm in computer input control. in Proceedings of the CHI '97 Conference on Human Factors in Computing Systems, (1997), 303--310.
[2]
Baudel, T. and Beaudouin-Lafon, M. Charade: remote control of objects using free-hand gestures. Communications of the ACM, 36 (7). 28--35.
[3]
Bowman, D., Wingrave, C., Campbell, J., Ly, V. and Rhoton, C. Novel uses of pinch gloves for virtual environment interaction techniques. Virtual Reality, 6 (3). 122--129.
[4]
Buxton, W., A Three-State Model of Graphical Input. in INTERACT '90, (1990), 449--456.
[5]
Hinckley, K., Czerwinski, M. and Sinclair, M., Interaction and modeling techniques for desktop two-handed input. in Proc. of the ACM UIST'98 Symposium on User Interface Software and Technology, (1998), 49--58.
[6]
Horn, B. K. P. Robot Vision. MIT Press, Cambridge, MA, 1986.
[7]
Krueger, M. W. Artificial Reality II. Addison-Wesley, Menlo Park, 1991.
[8]
Kurtenbach, G., Fitzmaurice, G., Baudel, T. and Buxton, W., The design and evaluation of a GUI paradigm based on two-hands tablets and transparency. in Proceedings of the CHI '97 Conference on Human Factors in Computing Systems, (1997).
[9]
MacKenzie, I. S. and Oniszczak, A., A comparison of three selection techniques for touchpads. in Proc. ACM CHI' 98 Conf. on Human Factors in Computing Systems, (1998), 336--343.
[10]
Malik, S. and Laszlo, J., Visual touchpad: a two-hand gestural input device. in Proceedings of the 6th International Conference on Multimodal Interfaces, (State College, PA, 2004), 289--296.
[11]
Quek, F., Mysliwiec, T. and Zhao, M., FingerMouse: A freehand pointing interface. in IEEE Automatic Face and Gesture Recognition, (1995), 372--377.
[12]
Wilson, A., PlayAnywhere: a compact tabletop computer vision system. in Proceedings of the 18th Annual ACM Symposium on User Interface Software Technology, (2005), 83--92.
[13]
Wilson, A. and Cutrell, E., FlowMouse: A computer vision-based pointing and gesture input device. in INTERACT '05, (Rome, Italy, 2005).

Cited By

View all
  • (2024)KeyFlow: Acoustic Motion Sensing for Cursor Control on Any KeyboardAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686348(1-3)Online publication date: 13-Oct-2024
  • (2023)Project Mihr: Enabling Gestural Interactions on a Keyboard using a Graphene-based FabricExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585873(1-7)Online publication date: 19-Apr-2023
  • (2023)PinchLens: Applying Spatial Magnification and Adaptive Control-Display Gain for Precise Selection in Virtual Reality2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00139(1221-1230)Online publication date: 16-Oct-2023
  • Show More Cited By

Index Terms

  1. Robust computer vision-based detection of pinching for one and two-handed gesture input

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      UIST '06: Proceedings of the 19th annual ACM symposium on User interface software and technology
      October 2006
      354 pages
      ISBN:1595933131
      DOI:10.1145/1166253
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 15 October 2006

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. bimanual interaction
      2. computer vision
      3. gesture
      4. hand tracking
      5. navigation

      Qualifiers

      • Article

      Conference

      UIST06

      Acceptance Rates

      Overall Acceptance Rate 561 of 2,567 submissions, 22%

      Upcoming Conference

      UIST '25
      The 38th Annual ACM Symposium on User Interface Software and Technology
      September 28 - October 1, 2025
      Busan , Republic of Korea

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)25
      • Downloads (Last 6 weeks)4
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)KeyFlow: Acoustic Motion Sensing for Cursor Control on Any KeyboardAdjunct Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3672539.3686348(1-3)Online publication date: 13-Oct-2024
      • (2023)Project Mihr: Enabling Gestural Interactions on a Keyboard using a Graphene-based FabricExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585873(1-7)Online publication date: 19-Apr-2023
      • (2023)PinchLens: Applying Spatial Magnification and Adaptive Control-Display Gain for Precise Selection in Virtual Reality2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00139(1221-1230)Online publication date: 16-Oct-2023
      • (2023)MyPGI - a methodology to yield personalized gestural interactionUniversal Access in the Information Society10.1007/s10209-022-00965-w23:2(795-820)Online publication date: 3-Jan-2023
      • (2023)Mixed Reality Interaction TechniquesSpringer Handbook of Augmented Reality10.1007/978-3-030-67822-7_5(109-129)Online publication date: 1-Jan-2023
      • (2022)Push or Pinch? Exploring Slider Control Gestures for Touchless User InterfacesNordic Human-Computer Interaction Conference10.1145/3546155.3546702(1-10)Online publication date: 8-Oct-2022
      • (2022)OmniLantern: Design and Implementations of a Portable and Coaxial Omnidirectional Projector-Camera SystemProceedings of the 2022 International Conference on Advanced Visual Interfaces10.1145/3531073.3531126(1-5)Online publication date: 6-Jun-2022
      • (2022)Investigating Cross-Modal Approaches for Evaluating Error Acceptability of a Recognition-Based Input TechniqueProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35172626:1(1-24)Online publication date: 29-Mar-2022
      • (2022)Overcoming Legacy Bias: Re-Designing Gesture Interactions in Virtual Reality With a San Community in NamibiaProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517549(1-18)Online publication date: 29-Apr-2022
      • (2022)Investigating Clutching Interactions for Touchless Medical Imaging SystemsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3517512(1-14)Online publication date: 29-Apr-2022
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media