Skip to main content
SpringerLink
Log in

CountMarks: Multi-finger Marking Menus for Mobile Interaction with Head-Mounted Displays

  • Conference paper
  • First Online:
Book cover HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12424))

Included in the following conference series:

Abstract

We designed, implemented, and evaluated CountMarks, a novel menu that extends marking menus with multi-touch input on a secondary touchscreen, for use with smart glasses and head-mounted displays. We experimentally compared CountMarks to marking menus. Results indicate that CountMarks offers faster selection and better search accuracy, but slightly worse selection accuracy. A second experiment compared standing vs. walking while using CountMarks, and handheld vs. on-leg interaction. Results indicate that CountMarks can be used effectively with a handheld device while both standing and walking. We present an example application employing CountMarks in a mock Netflix UI, to demonstrate how the technique can be applied in existing applications.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    For examples, see, e.g., https://www.nhtsa.gov/risky-driving/distracted-driving or https://news.osu.edu/distracted-walking-injuries-soar-for-pedestrians-on-phones/.

References

  1. Bailly, G.: Finger-count & radial-stroke shortcuts: two techniques for augmenting linear menus on multi-touch surfaces. In: Hand, pp. 1–4 (2010)

    Google Scholar 

  2. Bailly, G., Lecolinet, E., Nigay, L.: Visual menu techniques. ACM Comput. Surv. 49(4), 1–41 (2016)

    Article  Google Scholar 

  3. Bailly, G., Lecolinet, E., Nigay, L.: Flower menus. In: Proceedings of the Working Conference on Advanced Visual Interfaces – AVI 2008, p. 15 (2008)

    Google Scholar 

  4. Bailly, G., Müller, J., Lecolinet, E.: Design and evaluation of finger-count interaction: combining multitouch gestures and menus. Int. J. Hum. Comput. Stud. 70(10), 673–689 (2012)

    Article  Google Scholar 

  5. Bailly, G., Lecolinet, E., Nigay, L.: Wave menus: improving the novice mode of hierarchical marking menus. In: Baranauskas, C., Palanque, P., Abascal, J., Barbosa, S.D.J. (eds.) INTERACT 2007. LNCS, vol. 4662, pp. 475–488. Springer, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74796-3_45

    Chapter  Google Scholar 

  6. Cao, C.G.L., Zhou, M., Jones, D.B., Schwaitzberg, S.D.: Can surgeons think and operate with haptics at the same time? J. Gastrointest. Surg. 11(11), 1564–1569 (2007). https://doi.org/10.1007/s11605-007-0279-8

    Article  Google Scholar 

  7. Carroll, J.M., Rosson, M.B.: Paradox of the active user. In: Carroll, J.M. (ed.) Interfacing Thought: Cognitive Aspects of Human-Computer Interaction, pp. 80–111. MIT Press, Cambridge (1987)

    Google Scholar 

  8. Cockburn, A., Quinn, P., Gutwin, C., Ramos, G., Looser, J.: Air pointing: design and evaluation of spatial target acquisition with and without visual feedback. Int. J. Hum. Comput. Stud. 69(6), 401–414 (2011)

    Article  Google Scholar 

  9. Dobbelstein, D., Hock, P., Rukzio, E.: Belt: an unobtrusive touch input device for head-worn displays. In: Proceedings of the ACM CHI 2015 Conference on Human Factors in Computing Systems, vol. 1, pp. 2135–2138 (2015)

    Google Scholar 

  10. Harrison, C., Benko, H., Wilson, A.D.: OmniTouch. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology – UIST 2011, p. 441 (2011)

    Google Scholar 

  11. Harrison, C., Tan, D., Morris, D.: Skinput. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems – CHI 2010, p. 453 (2010)

    Google Scholar 

  12. Hincapié-Ramos, J.D., Guo, X., Moghadasian, P., Irani, P.: Consumed endurance. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems – CHI 2014, February, pp. 1063–1072 (2014)

    Google Scholar 

  13. Hsieh, Y.-T., Jylhä, A., Orso, V., Gamberini, L., Jacucci, G.: Designing a willing-to-use-in-public hand gestural interaction technique for smart glasses. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems – CHI 2016, pp. 4203–4215 (2016)

    Google Scholar 

  14. Kiger, J.I.: The depth/breadth trade-off in the design of menu-driven user interfaces. Int. J. Man Mach. Stud. 20(2), 201–213 (1984)

    Article  Google Scholar 

  15. Koelle, M., Kranz, M., Andreas, M.: Don’t look at me that way! – understanding user attitudes towards data glasses usage. In: MobileHCI 2015 Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Service, pp. 362–372 (2015)

    Google Scholar 

  16. Kulshreshth, A., LaViola, J.J.: Exploring the usefulness of finger-based 3D gesture menu selection. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems – CHI 2014, pp. 1093–1102 (2014)

    Google Scholar 

  17. Kurtenbach, G., Buxton, W.: The limits of expert performance using hierarchic marking menus. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems – CHI 1993, pp. 482–487 (1993)

    Google Scholar 

  18. Kurtenbach, G.: The design and evaluation of marking menus. University of Toronto, Toronto (1993)

    Google Scholar 

  19. Lane, D.M., Napier, H.A., Peres, S.C., Sandor, A.: Hidden costs of graphical user interfaces: failure to make the transition from menus and icon toolbars to keyboard shortcuts. Int. J. Hum. Comput. Interact. 18(2), 133–144 (2005)

    Article  Google Scholar 

  20. Lepinski, G.J., Grossman, T., Fitzmaurice, G.: The design and evaluation of multitouch marking menus. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems – CHI 2010, p. 2233 (2010)

    Google Scholar 

  21. Liu, M., Nancel, M., Vogel, D.: Gunslinger: Subtle Arms-down Mid-air Interaction, pp. 63–71 (2015)

    Google Scholar 

  22. Lucero, A., Vetek, A.: NotifEye. In: Proceedings of the 11th Conference on Advances in Computer Entertainment Technology – ACE 2014, pp. 1–10 (2014)

    Google Scholar 

  23. Pielot, M., Hesselmann, T., Heuten, W., Kazakova, A., Boll, S.: PocketMenu: non-visual menus for touch screen devices. In: Proceedings of the International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2013), pp. 327–330 (2012)

    Google Scholar 

  24. Ren, G., O’Neill, E.: 3D Marking menu selection with freehand gestures. In: IEEE Symposium on 3D User Interfaces 2012, 3DUI 2012 - Proceedings, pp. 61–68 (2012)

    Google Scholar 

  25. Saponas, T.S., Harrison, C., Benko, H.: PocketTouch. In: Proceedings of the 24th Annual ACM Symposium on User Interface Software and Technology – UIST 2011, Figure 1, p. 303 (2011)

    Google Scholar 

  26. Serrano, M., Ens, B.M., Irani, P.P.: Exploring the use of hand-to-face input for interacting with head-worn displays. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 3181–3190 (2014)

    Google Scholar 

  27. Shneiderman, B.: Direct manipulation: a step beyond programming languages (excerpt). In: Readings in Human-Computer Interaction: A Multidisciplinary Approach, pp. 461–467 (1987)

    Google Scholar 

  28. Teather, R.J., Stuerzlinger, W.: Pointing at 3D target projections with one-eyed and stereo cursors. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 159–168 (2013)

    Google Scholar 

  29. Tung, Y.-C., et al.: User-defined game input for smart glasses in public space. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems – CHI 2015, pp. 3327–3336 (2015)

    Google Scholar 

  30. Wang, C.-Y., et al.: PalmGesture: using palms as gesture interfaces for eyes-free input. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services, pp. 217–226 (2015)

    Google Scholar 

  31. Weigel, M., Mehta, V., Steimle, J.: More than touch: understanding how people use skin as an input surface for mobile computing. In: Proceedings of the 32nd Annual ACM Conference on Human Factors in Computing Systems – CHI 2014, pp. 179–188 (2014)

    Google Scholar 

  32. Weigel, M., Lu, T., Bailly, G., Oulasvirta, A., Majidi, C., Steimle, J.: iSkin: flexible, stretchable and visually customizable on-body touch sensors for mobile computing. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems – CHI 2015, pp. 2991–3000 (2015)

    Google Scholar 

  33. Weigel, M., Nittala, A.S., Olwal, A., Steimle, J.: SkinMarks: enabling interactions on body landmarks using conformal skin electronics. In: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems – CHI 2017, pp. 3095–3105 (2017)

    Google Scholar 

  34. Weiser, M.: The computer for 21st century. Sci. Am. Ubicomp Pap. 265(3), 8 (1991)

    Google Scholar 

  35. Yoon, S.H., Huo, K., Ramani, K.: Wearable textile input device with multimodal sensing for eyes-free mobile interaction during daily activities. Pervasive Mob. Comput. 33, 17–31 (2016)

    Article  Google Scholar 

  36. Zhang, Y., Zhou, J., Laput, G., Harrison, C.: SkinTrack: using the body as an electrical waveguide for continuous finger tracking on the skin. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems – CHI 2016, pp. 1491–1503 (2016)

    Google Scholar 

  37. Zhao, S., Balakrishnan, R.: Simple vs. compound mark hierarchical marking menus. In: Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology – UIST 2004, vol. 6, no. 2, p. 33 (2004)

    Google Scholar 

  38. Zhao, S., Agrawala, M., Hinckley, K.: Zone and polygon menus: using relative position to increase the breadth of multi-stroke marking menus. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1077–1086 (2006)

    Google Scholar 

  39. Zhao, S., Dragicevic, P., Chignell, M., Balakrishnan, R., Baudisch, P.: earPod: eyes-free menu selection using touch input and reactive audio feedback. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, p. 1395 (2007)

    Google Scholar 

  40. Zheng, J., Bi, X., Li, K., Li, Y., Zhai, S.: M3 gesture menu. In: Proceedings of the 2018 CHI Conference on Human Factors in Computing System – CHI 2018, pp. 1–14 (2018)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computer Science, Carleton University, Ottawa, ON, Canada

    Jordan Pollock

  2. School of Information Technology, Carleton University, Ottawa, ON, Canada

    Robert J. Teather

Authors
  1. Jordan Pollock
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Robert J. Teather
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robert J. Teather .

Editor information

Editors and Affiliations

  1. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

  2. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  3. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  4. University of Central Florida, Orlando, FL, USA

    Lauren Reinerman-Jones

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pollock, J., Teather, R.J. (2020). CountMarks: Multi-finger Marking Menus for Mobile Interaction with Head-Mounted Displays. In: Stephanidis, C., Kurosu, M., Degen, H., Reinerman-Jones, L. (eds) HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence. HCII 2020. Lecture Notes in Computer Science(), vol 12424. Springer, Cham. https://doi.org/10.1007/978-3-030-60117-1_18

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60117-1_18

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60116-4

  • Online ISBN: 978-3-030-60117-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation