skip to main content
10.1145/1936652.1936665acmconferencesArticle/Chapter ViewAbstractPublication PagesissConference Proceedingsconference-collections
research-article

Using a depth camera as a touch sensor

Published: 07 November 2010 Publication History

Abstract

We explore the application of depth-sensing cameras to detect touch on a tabletop. Limits of depth estimate resolution and line of sight requirements dictate that the determination of the moment of touch will not be as precise as that of more direct sensing techniques such as capacitive touch screens. However, using a depth-sensing camera to detect touch has significant advantages: first, the interactive surface need not be instrumented. Secondly, this approach allows touch sensing on non-flat surfaces. Finally, information about the shape of the users and their arms and hands above the surface may be exploited in useful ways, such as determining hover state, or that multiple touches are from same hand or from the same user. We present techniques and findings using Microsoft Kinect.

Supplementary Material

JPG File (p69-wilson.jpg)
MP4 File (p69-wilson.mp4)

References

[1]
http://www.xbox.com/kinect, last accessed Sep. 28, 2010.
[2]
http://www.primesense.com, last accessed Sep. 28, 2010.
[3]
Agarwal, A., Izadi, S., Chandraker M., Blake, A. High precision multi-touch sensing on surfaces using overhead cameras. Proc. IEEE Tabletop 2007, 197--200.
[4]
Benko, H., and Wilson, A. D. DepthTouch: using depth-sensing camera to enable freehand interactions on and above the interactive surface. Microsoft Research Technical Report MSR-TR-2009-23. March, 2009.
[5]
Dang, C. T., Straub, M., and André, E. Hand distinction for multi-touch tabletop interaction. Proc. ACM ITS 2009, 101--108.
[6]
Dietz, P. and Leigh, D. DiamondTouch: a multi-user touch technology. Proc. ACM UIST 2001, 219--226.
[7]
Hilliges, O., Izadi, S., Wilson, A. D., Hodges, S., Garcia-Mendoza, A., and Butz, A. 2009. Interactions in the air: adding further depth to interactive tabletops. Proc. ACM UIST 2009, 139--148.
[8]
Holz, C. and Baudisch, P. The generalized perceived input point model and how to double touch accuracy by extracting fingerprints. Proc. ACM CHI 2010. 581--590.
[9]
Kjeldsen, R., Pinhanez, C., Pingali, G., Jacob Hartman, J., Tony Levas, T., Podlaseck, M. Interacting with steerable projected displays. Proc. International Conference on Automatic Face and Gesture Recognition, 2002.
[10]
Krueger, M. Artificial Reality 2, Addison-Wesley, 1991.
[11]
Letessier, J. and Bérard, F. Visual tracking of bare fingers for interactive surfaces. Proc. ACM UIST 2004, 119--122.
[12]
Malik, S. and Laszlo, J. Visual touchpad: a two-handed gestural input device. Proc. ICMI 2004. 289--296.
[13]
Marshall, J., Pridmore, T., Pound, M., Benford, S., and Koleva, B. Pressing the Flesh: Sensing Multiple Touch and Finger Pressure on Arbitrary Surfaces. Proc. Pervasive 2008, 38--55.
[14]
Wang, F., Cao, X., Ren, X., and Irani, P. Detecting and leveraging finger orientation for interaction with direct-touch surfaces. Proc. ACM UIST 2009, 23--32.
[15]
Wellner, P. The DigitalDesk calculator: tangible manipulation on a desk top display. Proc. ACM UIST 1991. 27--33.
[16]
Wilson, A. D. TouchLight: an imaging touch screen and display for gesture-based interaction. Proc. ICMI 2004, 69--76.
[17]
Wilson, A. D. PlayAnywhere: a compact interactive tabletop projection-vision system. Proc. ACM UIST 2005. 83--92.
[18]
Wilson, A. D. Depth sensing video cameras for 3D tangible tabletop interaction. Proc. IEEE Tabletop 2007. 201--204.
[19]
Wilson, A. D., and Benko, H. Combing multiple depth cameras and projectors for interactions, on, above and between surfaces. Proc. ACM UIST 2010.
[20]
Wren, C., and Ivanov, Y. Volumetric operations with surface margins. In CVPR: Technical Sketches, 2001.

Cited By

View all
  • (2024)Capacitive Touch Sensing on General 3D SurfacesACM Transactions on Graphics10.1145/365818543:4(1-20)Online publication date: 19-Jul-2024
  • (2024)RFTIRTouch: Touch Sensing Device for Dual-sided Transparent Plane Based on Repropagated Frustrated Total Internal ReflectionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676428(1-10)Online publication date: 13-Oct-2024
  • (2024)TouchInsight: Uncertainty-aware Rapid Touch and Text Input for Mixed Reality from Egocentric VisionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676330(1-16)Online publication date: 13-Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ITS '10: ACM International Conference on Interactive Tabletops and Surfaces
November 2010
327 pages
ISBN:9781450303996
DOI:10.1145/1936652
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 November 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. depth-sensing cameras
  2. touchscreen interfaces

Qualifiers

  • Research-article

Conference

ITS'10
Sponsor:
ITS'10: Interactive Tabletops and Surfaces
November 7 - 10, 2010
Saarbrücken, Germany

Acceptance Rates

Overall Acceptance Rate 119 of 418 submissions, 28%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)79
  • Downloads (Last 6 weeks)5
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Capacitive Touch Sensing on General 3D SurfacesACM Transactions on Graphics10.1145/365818543:4(1-20)Online publication date: 19-Jul-2024
  • (2024)RFTIRTouch: Touch Sensing Device for Dual-sided Transparent Plane Based on Repropagated Frustrated Total Internal ReflectionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676428(1-10)Online publication date: 13-Oct-2024
  • (2024)TouchInsight: Uncertainty-aware Rapid Touch and Text Input for Mixed Reality from Egocentric VisionProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology10.1145/3654777.3676330(1-16)Online publication date: 13-Oct-2024
  • (2024)TriPad: Touch Input in AR on Ordinary Surfaces with Hand Tracking OnlyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642323(1-18)Online publication date: 11-May-2024
  • (2024)Touch Interaction Classification Using Upper-Body Sensor SuitIEEE Sensors Journal10.1109/JSEN.2024.340710624:14(22720-22732)Online publication date: 15-Jul-2024
  • (2024)RGB-D Camera-Based Object Grounding Surface Estimation System2024 IEEE 21st Consumer Communications & Networking Conference (CCNC)10.1109/CCNC51664.2024.10454727(586-589)Online publication date: 6-Jan-2024
  • (2023)TouchPilot: Designing a Guidance System that Assists Blind People in Learning Complex 3D StructuresProceedings of the 25th International ACM SIGACCESS Conference on Computers and Accessibility10.1145/3597638.3608426(1-18)Online publication date: 22-Oct-2023
  • (2023)Structured Light Speckle: Joint Ego-Centric Depth Estimation and Low-Latency Contact Detection via Remote VibrometryProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606749(1-12)Online publication date: 29-Oct-2023
  • (2023)The tactile dimension: a method for physicalizing touch behaviorsProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581137(1-15)Online publication date: 19-Apr-2023
  • (2022)Reducing the Latency of Touch Tracking on Ad-hoc SurfacesProceedings of the ACM on Human-Computer Interaction10.1145/35677306:ISS(489-499)Online publication date: 14-Nov-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media