skip to main content
10.1145/1731903.1731933acmconferencesArticle/Chapter ViewAbstractPublication PagesissConference Proceedingsconference-collections
research-article

Investigating multi-touch and pen gestures for diagram editing on interactive surfaces

Published: 23 November 2009 Publication History

Abstract

Creating and editing large graphs and node-link diagrams are crucial activities in many application areas. For them, we consider multi-touch and pen input on interactive surfaces as very promising. This fundamental work presents a user study investigating how people edit node-link diagrams on an interactive tabletop. The study covers a set of basic operations, such as creating, moving, and deleting diagram elements. Participants were asked to perform spontaneous gestures for 14 given tasks. They could interact in three different ways: using one hand, both hands, as well as pen and hand together. The subjects' activities were observed and recorded in various ways, analyzed and enriched with think-aloud data. As a result, we contribute a user-elicited collection of touch and pen gestures for editing node-link diagrams. The study provides valuable insight how people would interact on interactive surfaces for this as well as other tabletop domains.

Supplementary Material

JPG File (165.jpg)
MOV File (165.mov)

References

[1]
Brandl, P.; Haller, M.; Hurnaus, M.; Lugmayr, V.; Oberngruber, J.; Oster, C.; Schafleitner, C. & Billinghurst, M. An Adaptable Rear-Projection Screen Using Digital Pens And Hand Gestures, In Proc ICAT 2007, IEEE Computer Society (2007), 49--54.
[2]
Brandl, P., Forlines, C., Wigdor, D., Haller, M., and Shen, C. Combining and measuring the benefits of bimanual pen and direct-touch interaction on horizontal interfaces. In Proc. AVI 2008, ACM Press (2008), 154--161.
[3]
Cao, X., Wilson, A. D., Balakrishnan, R., Hinckley, K., Hudson S. E. ShapeTouch: Leveraging Contact Shape on Interactive Surfaces. In Proc. of TABLETOP 2008, 139--146.
[4]
Chen, Q., Grundy, J., and Hosking, J. An e-whiteboard application to support early design-stage sketching of UML diagrams. In Proc. HCC 2003, IEEE Computer Society (2003), 219--226.
[5]
Cherubini, M., Venolia, G., DeLine, R., and Ko, A. J. Let's go to the whiteboard: how and why software developers use drawings. In Proc. CHI 2007, ACM Press (2007), 557--566.
[6]
Damm, C. H., Hansen, K. M., and Thomsen, M. Tool support for cooperative object-oriented design: gesture based modeling on an electronic whiteboard. In Proc. CHI 2000, ACM Press (2000), 518--525.
[7]
Dekel, U. Supporting distributed software design meetings: what can we learn from co-located meetings? In Proc. HSSE 2005, ACM Press (2005), 1--7.
[8]
Dietz, P. and Leigh, D. DiamondTouch: a multi-user touch technology. In Proc UIST 2001, ACM Press (2001), 219--226.
[9]
Epps, J.; Lichman, S. & Wu, M. A study of hand shape use in tabletop gesture interaction, Ext. Abstracts CHI 2006, ACM Press (2006), 748--753.
[10]
Grundy, J. and Hosking, J. Supporting Generic Sketching-Based Input of Diagrams in a Domain-Specific Visual Language Meta-Tool. In Pro. ICSE 2007, IEEE Computer Society (2007), 282--291.
[11]
Guiard, Y. Asymmetric Division of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model, The Journal of Motor Behavior 19, 4 (1987), 486--517.
[12]
Han, J. Y. Low-cost multi-touch sensing through frustrated total internal reflection. In Proc. UIST 2005, ACM Press, (2005), 115--118.
[13]
Leitner, J.; Powell, J.; Brandl, P.; Seifried, T.; Haller, M.; Dorray, B. & To, P. Flux: a tilting multi-touch and pen based surface, Ext. Abstracts CHI 2009, (2009), 3211--3216.
[14]
Malik, S., Ranjan, A., and Balakrishnan, R. Interacting with large displays from a distance with vision-tracked multi-finger gestural input. In Proc. UIST 2005, ACM Press (2005), 43--52.
[15]
Moran, T. P., Chiu, P., and van Melle, W. Pen-based interaction techniques for organizing material on an electronic whiteboard. In Proc. UIST 1997, ACM Press (1997), 45--54.
[16]
Nielsen, M., Störring, M., Moeslund, T. B. and Granum, E. A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In Proc. Int'l Gesture Workshop 2003, Springer Verlag (2004), 409--420.
[17]
Rekimoto, J. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In Proc. CHI 2002, ACM Press (2002), 113--120.
[18]
Ringel, M.; Ryall, K.; Shen, C.; Forlines, C. & Vernier, F. Release, relocate, reorient, resize: fluid techniques for document sharing on multi-user interactive tables, Ext. Abstracts CHI 2004, ACM Press (2004), 1441--1444.
[19]
Shen, C.; Vernier, F. D.; Forlines, C. & Ringel, M. DiamondSpin: an extensible toolkit for around-the-table interaction, In Proc. CHI 2004, ACM Press (2004), 167--174.
[20]
Wobbrock, J. O., Morris, M. R., and Wilson, A. D. User-defined gestures for surface computing. In Proc. CHI 2009. ACM Press (2009), 1083--1092.
[21]
Wu, M. & Balakrishnan, R. Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. In Proc. UIST 2003, ACM Press, (2003), 193--202.
[22]
Wu, M.; Shen, C.; Ryall, K.; Forlines, C. & Balakrishnan, R. Gesture Registration, Relaxation, and Reuse for Multi-Point Direct-Touch Surfaces, In Proc. TABLETOP '06, IEEE Computer Society, 185--192.
[23]
Yee, K. Two-handed interaction on a tablet display. Ext. Abstracts CHI 20004, ACM Press (2004), 1493--149

Cited By

View all
  • (2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
  • (2023)Comparing alternative modalities in the context of multimodal human–robot interactionJournal on Multimodal User Interfaces10.1007/s12193-023-00421-w18:1(69-85)Online publication date: 19-Oct-2023
  • (2022)Theoretically-Defined vs. User-Defined Squeeze GesturesProceedings of the ACM on Human-Computer Interaction10.1145/35678056:ISS(73-102)Online publication date: 14-Nov-2022
  • Show More Cited By

Index Terms

  1. Investigating multi-touch and pen gestures for diagram editing on interactive surfaces

          Recommendations

          Comments

          Information & Contributors

          Information

          Published In

          cover image ACM Conferences
          ITS '09: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces
          November 2009
          240 pages
          ISBN:9781605587332
          DOI:10.1145/1731903
          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Sponsors

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          Published: 23 November 2009

          Permissions

          Request permissions for this article.

          Check for updates

          Author Tags

          1. bimanual input
          2. diagram editing
          3. hand gestures
          4. multi-touch
          5. node-link diagrams
          6. pen interaction
          7. tabletop

          Qualifiers

          • Research-article

          Funding Sources

          Conference

          ITS'09
          Sponsor:

          Acceptance Rates

          Overall Acceptance Rate 119 of 418 submissions, 28%

          Contributors

          Other Metrics

          Bibliometrics & Citations

          Bibliometrics

          Article Metrics

          • Downloads (Last 12 months)30
          • Downloads (Last 6 weeks)1
          Reflects downloads up to 16 Feb 2025

          Other Metrics

          Citations

          Cited By

          View all
          • (2023)Brave New GES World: A Systematic Literature Review of Gestures and Referents in Gesture Elicitation StudiesACM Computing Surveys10.1145/363645856:5(1-55)Online publication date: 7-Dec-2023
          • (2023)Comparing alternative modalities in the context of multimodal human–robot interactionJournal on Multimodal User Interfaces10.1007/s12193-023-00421-w18:1(69-85)Online publication date: 19-Oct-2023
          • (2022)Theoretically-Defined vs. User-Defined Squeeze GesturesProceedings of the ACM on Human-Computer Interaction10.1145/35678056:ISS(73-102)Online publication date: 14-Nov-2022
          • (2022)Iteratively Designing Gesture Vocabularies: A Survey and Analysis of Best Practices in the HCI LiteratureACM Transactions on Computer-Human Interaction10.1145/350353729:4(1-54)Online publication date: 5-May-2022
          • (2022)Reducing the Cognitive Load of Playing a Digital Tabletop Game with a Multimodal InterfaceProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502062(1-13)Online publication date: 29-Apr-2022
          • (2022)Iterative Design of Gestures During Elicitation: Understanding the Role of Increased ProductionProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501962(1-14)Online publication date: 29-Apr-2022
          • (2021)Responsive Matrix Cells: A Focus+Context Approach for Exploring and Editing Multivariate GraphsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.303037127:2(1644-1654)Online publication date: Feb-2021
          • (2021)Interweaving Multimodal Interaction With Flexible Unit Visualizations for Data ExplorationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.297805027:8(3519-3533)Online publication date: 1-Aug-2021
          • (2021)The Study of Model for Two-Handed Pointing Tasks in Pen + Touch InterfacesIEEE Access10.1109/ACCESS.2021.30638829(38087-38096)Online publication date: 2021
          • (2021)Deep Learning-Based Hand Posture Recognition for Pen Interaction EnhancementArtificial Intelligence for Human Computer Interaction: A Modern Approach10.1007/978-3-030-82681-9_7(193-225)Online publication date: 5-Nov-2021
          • Show More Cited By

          View Options

          Login options

          View options

          PDF

          View or Download as a PDF file.

          PDF

          eReader

          View online with eReader.

          eReader

          Figures

          Tables

          Media

          Share

          Share

          Share this Publication link

          Share on social media