skip to main content
10.1145/1095034.1095042acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
Article

Interacting with large displays from a distance with vision-tracked multi-finger gestural input

Published: 23 October 2005 Publication History

Abstract

We explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. We develop bimanual techniques to support a variety of asymmetric and symmetric interactions, including fast targeting and navigation to all parts of a large display from the comfort of a desk and chair, as well as techniques that exploit the ability of the vision-based hand tracking system to provide multi-finger identification and full 2D hand segmentation. We also posit a design that allows for handling multiple concurrent users.

References

[1]
Balakrishnan, R., and MacKenzie, I. S. (1997). Performance Differences in the Fingers, Wrist, and Forearm in Computer Input Control. In Proceedings of ACM CHI Conference. p. 303--310.
[2]
Baudisch, P., Cutrell, E., Robbins, D., Czerwinski, M., Tandler, P. Bederson, B., and Zierlinger, A. (2003). Drag-and-Pop and Drag-and-Pick: Techniques for Accessing Remote Screen Content on Touch- and Pen-operated Systems. In Proceedings of Interact. p. 57--64.
[3]
Bezerianos, A and Balakrishnan, R. (2005 -- in press). The Vacuum: Facilitating the Manipulation of Distant Objects. In Proceedings of ACM CHI Conference.
[4]
Bier, E. A., Stone, M. C., Pier, K., Buxton, W., DeRose, T. D. (1993). Toolglass and Magic Lenses: The See-through Interface. In Proceedings of ACM SIGGRAPH. p. 73--80.
[5]
Cao, X., and Balakrishnan, R. (2003). VisionWand: Interaction Techniques for Large Displays Using a Passive Wand Tracked in 3D. In Proceedings of ACM UIST Symposium. p. 173--182.
[6]
Davis, J., Chen, X. (2002). Lumipoint: Multi-User Laser-Based Interaction on Large Tiled Displays. In Displays, Volume 23, Issue 5.
[7]
Dietz, P., and Leigh, D. (2001). DiamondTouch: A Multi-user Touch Technology. In Proceedings of ACM UIST Symposium. p. 219--226.
[8]
Fiala, M. (2004). ARTag Revision 1, A Fiducial Marker System Using Digital Techniques. NRC/ERB-1117 Technical Report. National Research Council of Canada.
[9]
Geißler, J. (1998). Shuffle, Throw or Take It! Working Efficiently with an Interactive Wall. (1998). In ACM CHI Conference, Extended Abstracts. p. 265--266.
[10]
Graham, E., MacKenzie, C. (1996). Physical versus Virtual Pointing. In Proceedings of ACM CHI Conference. p. 292--299.
[11]
Guiard, Y. (1987). Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. In Journal of Motor Behavior, 19, p. 486--517.
[12]
Guimbretière, F., Stone, M., Winograd, T. (2001). Fluid Interaction with High-resolution Wall-size Displays. In Proceedings of ACM UIST Symposium. p. 21--30.
[13]
Hardenberg, C. V., Bérard, F. (2001). Bare-Hand Human-Computer Interaction. In Proceedings of the Workshop on Perceptive User Interface.
[14]
Hinckley, K., Pausch, R., Proffitt, D., Patten, J., Kassell, N. (1997). Cooperative Bimanual Action. In Proceedings of ACM CHI Conference. p. 27--34.
[15]
Humphreys, G., Houston, M., Ng, R., Frank, R., et al. (2002). Chromium: A Stream-Processing Framework for Interactive Rendering on Clusters. In ACM Transactions on Graphics, Proceedings of ACM SIGGRAPH. p. 693--702.
[16]
Izadi, S., Brignull, H., Rodden, T., Rogers, Y., Underwood, M. (2003). Dynamo: A Public Interactive Surface Supporting the Cooperative Sharing and Exchange of Media. In Proceedings of ACM UIST Symposium. p. 159--168.
[17]
Johanson, B., Fox, A., and Winograd, T. (2002). The Interactive Workspace Project: Experiences with Ubiquitous Computing Rooms. In IEEE Pervasive Computing. p. 67--74.
[18]
Johanson, B., Hutchins, G., Winograd, T., Stone, M. (2002). Pointright: Experience with Flexible Input Redirection in Interactive Workspaces. In Proceedings of ACM UIST Symposium. p. 227--234.
[19]
Kabbash, P., Buxton, W., and Sellen, A. (1994). Two-handed Input in a Compound Task. In Proceedings of ACM CHI Conference. p. 17--423.
[20]
Khan, A., Matejka, J., Fitzmaurice, G., Kurtenbach, G. (2005 -- in press). Spotlight: Directing Users' Visual Attention on Large Displays. In Proceedings of the ACM CHI Conference.
[21]
Khan, A., Fitzmaurice, G., Almeida, D., Burtnyk, N., Kurtenbach, G. (2004). A Remote Control Interface for Large Displays. In Proceedings of ACM UIST Symposium. p. 127--136.
[22]
Kirsh, D., & Maglio, P. (1994). On Distinguishing Epistemic from Pragmatic action. In Cognitive Science, 18. p. 513--549.
[23]
Kirstein, C. and Muller, H. (1998). Interaction with a Projection Screen using a Camera-tracked Laser Pointer. In Proceedings of Multimedia Modeling. p. 191.
[24]
Kjeldsen, R., and Hartman, J. (2001). Design Issues for Vision-based Computer Interaction Systems. In Proceedings of the Workshop on Perceptive User Interfaces.
[25]
Kurtenbach, G., Fitzmaurice, G., Baudel, T., Buxton, W. (1997). The Design of a GUI Paradigm based on Tablets, Two Hands, and Transparency. In Proceedings of ACM CHI Conference. p. 35--42.
[26]
Malik, S. and Laszlo, J. (2004). Visual Touchpad: A Two-handed Gestural Input Device. In Proceedings of the ACM International Conference on Multimodal Interfaces. p. 289--296.
[27]
Mynatt, E., Igarashi, T., Edwards, W., LaMarca, A. (1999). Flatland: New Dimensions in Office Whiteboards. In Proceedings of ACM CHI Conference. p. 346--353.
[28]
Pederson, E., McCall, K., Moran, T., Halasz, F. (1993). Tivoli: An Electronic Whiteboard for Informal Workgroup Meetings. In Proceedings of ACM CHI Conference. p. 391--398.
[29]
Rekimoto, J. (2002). SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. In Proceedings of ACM CHI Conference. p. 113--120.
[30]
Swaminathan, K, and Sato, S. (1997). Interaction Design for Large Displays. Interactions, Volume 4, Issue 1.
[31]
Tactiva. (2005). TactaPad. http://www.tactiva.com.
[32]
Wellner, P. (1993). Interacting with Paper on the DigitalDesk. In Communications of the ACM, Volume 36, Issue 7. p. 87--96.
[33]
Wilson, A. (2004). Touchlight: An Imaging Touch Screen and Display for Gesture-Based Interaction. In Proceedings of the International Conference on Multimodal Interfaces. p. 69--76.
[34]
Wu, M., and Balakrishnan, R. (2003). Multi-finger and Whole Hand Gestural Interaction Techniques for Multi-user Tabletop Displays. In Proceedings of ACM UIST Symposium. p. 193--202.

Cited By

View all
  • (2024)Interactive Visualization on Large High‐Resolution Displays: A SurveyComputer Graphics Forum10.1111/cgf.1500143:6Online publication date: 30-Apr-2024
  • (2024)Submerse: Visualizing Storm Surge Flooding Simulations in Immersive Display EcologiesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.333251130:9(6365-6377)Online publication date: Sep-2024
  • (2024)A Gesture Recognition System Based On MEMS Accelerometer2024 36th Chinese Control and Decision Conference (CCDC)10.1109/CCDC62350.2024.10587430(3987-3992)Online publication date: 25-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technology
October 2005
270 pages
ISBN:1595932712
DOI:10.1145/1095034
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2005

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. asymmetric
  2. bimanual
  3. from afar
  4. gesture
  5. interaction
  6. large wall
  7. multi-point
  8. symmetric
  9. touch surface
  10. two hands
  11. visual touchpad

Qualifiers

  • Article

Conference

UIST05

Acceptance Rates

UIST '05 Paper Acceptance Rate 31 of 159 submissions, 19%;
Overall Acceptance Rate 561 of 2,567 submissions, 22%

Upcoming Conference

UIST '25
The 38th Annual ACM Symposium on User Interface Software and Technology
September 28 - October 1, 2025
Busan , Republic of Korea

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)26
  • Downloads (Last 6 weeks)2
Reflects downloads up to 24 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Interactive Visualization on Large High‐Resolution Displays: A SurveyComputer Graphics Forum10.1111/cgf.1500143:6Online publication date: 30-Apr-2024
  • (2024)Submerse: Visualizing Storm Surge Flooding Simulations in Immersive Display EcologiesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.333251130:9(6365-6377)Online publication date: Sep-2024
  • (2024)A Gesture Recognition System Based On MEMS Accelerometer2024 36th Chinese Control and Decision Conference (CCDC)10.1109/CCDC62350.2024.10587430(3987-3992)Online publication date: 25-May-2024
  • (2022)Asymmetric Free-Hand Interaction on a Large Display and Inspirations for Designing Natural User InterfacesSymmetry10.3390/sym1405092814:5(928)Online publication date: 2-May-2022
  • (2021)VXSlate: Exploring Combination of Head Movements and Mobile Touch for Large Virtual Display InteractionProceedings of the 2021 ACM Designing Interactive Systems Conference10.1145/3461778.3462076(283-297)Online publication date: 28-Jun-2021
  • (2020)Designing for conflictProceedings of the 9TH ACM International Symposium on Pervasive Displays10.1145/3393712.3395336(47-54)Online publication date: 4-Jun-2020
  • (2020)COMTIS: Customizable touchless interaction system for large screen visualizationVirtual Reality & Intelligent Hardware10.1016/j.vrih.2020.01.0032:2(162-174)Online publication date: Apr-2020
  • (2020)Ocean Data Visualization on a Touchtable Demonstrates Group Content Learning, Science Practices Use, and Potential Embodied CognitionResearch in Science Education10.1007/s11165-020-09951-952:2(445-457)Online publication date: 6-Aug-2020
  • (2020)Using large-scale augmented floor surfaces for industrial applications and evaluation on perceived sizesPersonal and Ubiquitous Computing10.1007/s00779-020-01433-z26:3(721-736)Online publication date: 11-Aug-2020
  • (2020)Multi-person Spatial Interaction in a Large Immersive Display Using Smartphones as TouchpadsIntelligent Systems and Applications10.1007/978-3-030-55190-2_22(285-302)Online publication date: 25-Aug-2020
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media