skip to main content
10.1145/1027933.1027980acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
Article

Visual touchpad: a two-handed gestural input device

Published: 13 October 2004 Publication History

Abstract

This paper presents the Visual Touchpad, a low-cost vision-based input device that allows for fluid two-handed interactions with desktop PCs, laptops, public kiosks, or large wall displays. Two downward-pointing cameras are attached above a planar surface, and a stereo hand tracking system provides the 3D positions of a user's fingertips on and above the plane. Thus the planar surface can be used as a multi-point touch-sensitive device, but with the added ability to also detect hand gestures hovering above the surface. Additionally, the hand tracker not only provides positional information for the fingertips but also finger orientations. A variety of one and two-handed multi-finger gestural interaction techniques are then presented that exploit the affordances of the hand tracker. Further, by segmenting the hand regions from the video images and then augmenting them transparently into a graphical interface, our system provides a compelling direct manipulation experience without the need for more expensive tabletop displays or touch-screens, and with significantly less self-occlusion.

References

[1]
Buxton, W. (1992). Telepresence: integrating shared task and person spaces. In Proceedings of Graphics Interface, pp. 123--129.]]
[2]
Chen, X., Koike, H., Nakanishi, Y., Oka, K., Sato, Y. (2002). Two-handed drawing on augmented desk system. In Proceedings of Advanced Visual Interfaces (AVI). pp. 219--222.]]
[3]
Corso, J., Burschka, D., Hager, G. (2003). The 4D Touchpad: Unencumbered HCI with VICs. In Proceedings of CVPR-HCI.]]
[4]
Faugeras, O., Luong, Q. (2001). The Geometry of Multiple Images. The MIT Press.]]
[5]
Guiard, Y. (1987). Asymmetric Divison of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. Journal of Motor Behavior, 19(4). pp. 486--517.]]
[6]
Hardenberg, C., Berard, F. (2001). Bare-hand Human-computer Interaction. In Proceedings of ACM Workshop on Perceptive User Interfaces (PUI).]]
[7]
Ishii, H., Kobayashi, M. (1992). ClearBoard: a seamless medium for shared drawing and conversation with eye contact. In Proceedings of ACM CHI Conference. pp. 525--532.]]
[8]
Kabbash, P., Buxton, W., Sellen, A. (1994). Two-Handed Input in a Compound Task. In Proceedings of ACM CHI Conference. pp. 417--423.]]
[9]
Kolsch, M., Turk, M. (2002). Keyboards without Keyboards: A Survey of Virtual Keyboards. Technical Report 2002-21. University of California, Santa Barbara.]]
[10]
Krueger, M. (1991). VIDEOPLACE and the Interface of the Future. In The Art of Human Computer Interface Design. Addison Wesley, Menlo Park, CA. pp. 417--422.]]
[11]
Kurtenbach, G., Fitzmaurice, G., Baudel, T., Buxton, B. (1997). The Design of a GUI Paradigm based on Tablets, Two-hands, and Transparency. In Proceedings of ACM CHI Conference. pp. 35--42.]]
[12]
MacCormick, J., Isard, M. (2000). Partitioned sampling, articulated objects, and interface-quality hand tracking. In Proceedings of the European Conference on Computer Vision (ECCV), Volume 2. pp. 3--19.]]
[13]
Mysliwiec, T. (1994). FingerMouse: A Freehand Computer Pointing Interface. Technical Report VISLab-94-001, University of Illinois Chicago.]]
[14]
Oka, K., Sato, Y., & Koike, H. (2002). Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems. In Proceedings of IEEE Conference on Automatic Face and Gesture Recognition (FG). pp. 429--434.]]
[15]
Rekimoto, J. (2002). SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. In Proceedings of ACM CHI Conference. pp. 113--120.]]
[16]
Ringel, M., Berg, H., Jin, Y., and Winograd, T. (2001). Barehands: Implement-Free Interaction with a Wall-Mounted Display. In Proceedings of ACM CHI Conference Extended Abstracts. pp. 367--368.]]
[17]
Roussel, N. (2001). Exploring new uses of video with videoSpace. In Proceedings of the IFIP Conference on Engineering for HCI, Volume 2254 of Lecture Notes in Computer Science, Springer. pp. 73--90.]]
[18]
Segen, J., & Kumar, S. (1998). GestureVR: Vision-based 3D Hand Interface for Spatial Interaction. In Proceedings of the Sixth ACM International Conference on Multimedia. pp. 455--464.]]
[19]
Stotts, D., Smith, J., and Jen, D. (2003). The Vis-a-Vid Transparent Video FaceTop. In Proceedings of ACM UIST. pp. 57--58.]]
[20]
Tang, J. & Minneman, S. (1991). Videowhiteboard: video shadows to support remote collaboration. In Proceedings of ACM CHI Conference. pp. 315--322.]]
[21]
Trucco, E., Verri, A. (1998). Introductory Techniques for 3-D Computer Vision. Prentice-Hall.]]
[22]
Ukita, N., Kidode, M. (2004). Wearable Virtual Tablet: Fingertip Drawing on a Portable Plane-Object using an Active-Infrared Camera. In Proceedings of the International Conference on Intelligent User Interfaces. pp. 169--176.]]
[23]
Wellner, P. (1993). Interacting with Paper on the DigitalDesk. Communications of the ACM, 36(7), July 1993. pp. 86--96.]]
[24]
Wu, M., Balakrishnan, R. (2003). Multi-finger and Whole Hand Gestural Interaction Techniques for Multi-User Tabletop Displays. In Proceedings ACM UIST. pp. 193--202.]]
[25]
Yee, K. (2004). Two-Handed Interaction on a Tablet Display. In Proceedings of ACM CHI Extended Abstracts. pp. 1493--1496.]]
[26]
Zhang, Z., Wu, Y., Shan, Y., & Shafer, S. (2001) Visual Panel: Virtual Mouse, Keyboard, and 3D Controller with an Ordinary Piece of Paper. In Proceedings of ACM Workshop on Perceptive User Interfaces (PUI).]]

Cited By

View all
  • (2024)TriPad: Touch Input in AR on Ordinary Surfaces with Hand Tracking OnlyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642323(1-18)Online publication date: 11-May-2024
  • (2024)Gesture centric interaction: evaluating hand and head gestures in touchless cursor controlErgonomics10.1080/00140139.2024.2411302(1-21)Online publication date: 23-Oct-2024
  • (2023)Vehicle Control on an Uninstrumented Surface With an Off-the-Shelf SmartwatchIEEE Transactions on Intelligent Vehicles10.1109/TIV.2023.32364588:5(3366-3374)Online publication date: May-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '04: Proceedings of the 6th international conference on Multimodal interfaces
October 2004
368 pages
ISBN:1581139950
DOI:10.1145/1027933
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 October 2004

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. augmented reality
  2. computer vision
  3. direct manipulation
  4. fluid interaction
  5. gestures
  6. hand tracking
  7. perceptual user interface
  8. two hand
  9. virtual keyboard
  10. virtual mouse
  11. visual touchpad

Qualifiers

  • Article

Conference

ICMI04
Sponsor:

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)39
  • Downloads (Last 6 weeks)3
Reflects downloads up to 24 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)TriPad: Touch Input in AR on Ordinary Surfaces with Hand Tracking OnlyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642323(1-18)Online publication date: 11-May-2024
  • (2024)Gesture centric interaction: evaluating hand and head gestures in touchless cursor controlErgonomics10.1080/00140139.2024.2411302(1-21)Online publication date: 23-Oct-2024
  • (2023)Vehicle Control on an Uninstrumented Surface With an Off-the-Shelf SmartwatchIEEE Transactions on Intelligent Vehicles10.1109/TIV.2023.32364588:5(3366-3374)Online publication date: May-2023
  • (2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
  • (2022)A Case for Precise, Fine-Grained Pointer Synthesis in High-Level SynthesisACM Transactions on Design Automation of Electronic Systems10.1145/349143027:4(1-26)Online publication date: 8-Mar-2022
  • (2022)Three-Dimensional Input System by Pinching Gestures for Vehicle DesignProceedings of the 38th International MATADOR Conference10.1007/978-3-319-64943-6_3(35-45)Online publication date: 27-Mar-2022
  • (2022)IoT‐Based Health Monitoring System for Speech‐Impaired People Using Assistive Wearable AccelerometerAdvanced Healthcare Systems10.1002/9781119769293.ch7(81-99)Online publication date: 29-Jan-2022
  • (2021)A Design Space for User Interface Elements using Finger Orientation InputProceedings of Mensch und Computer 202110.1145/3473856.3473862(1-10)Online publication date: 5-Sep-2021
  • (2021)Phonetroller: Visual Representations of Fingers for Precise Touch Input with Mobile Phones in VRProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445583(1-13)Online publication date: 6-May-2021
  • (2021)Virtual Mouse using Hand Gestures2021 International Conference on Technological Advancements and Innovations (ICTAI)10.1109/ICTAI53825.2021.9673251(340-345)Online publication date: 10-Nov-2021
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media