Christopher Ackad
Andrew Clayphan
Judy Kay
ABSTRACT
This paper describes the design and evaluation of our Media Ribbon, a large public interactive display for browsing hierarchical information, with mid-air gestures. Browsing a hierarchical information space is a fundamental form of interaction. Designing learnable mid-air gestures is a current challenge for large display interaction. Our in-the-wild evaluation draws on 41 days of quantitative log data, with 4484 gestures detected, and qualitative data from 15 interviews, and associated video. We explored: whether our design enabled people to learn the gestures; how our tutorial and feedback mechanisms supported learning; and the effectiveness of support for browsing hierarchical information. Our contributions are: (1) design of large public display for browsing of hierarchical information; (2) with its gesture set; (3) insights into the ways people learn and use this interface in our context; and (4) guidelines for designing learnable mid-air gestures.
Supplemental Material
- Ackad, C., Wasinger, R., Gluga, R., Kay, J., and Tomitsch, M. Measuring Interactivity at an Interactive Public Information Display. In the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, Collaboration, OzCHI '13, ACM (2013), 329--332. Google Scholar
Digital Library
- Aigner, R., Wigdor, D., Benko, H., Haller, M., Lindlbauer, D., Ion, A., Zhao, S., Koh, J. T. K. V., and Redmond, W. A. Understanding mid-air hand gestures: A study of human preferences in usage of gesture types for HCI. Microsoft Research TechReport MSR-TR-2012-111 (2012).Google Scholar
- Akpan, I., Marshall, P., Bird, J., and Harrison, D. Exploring the effects of space and place on engagement with an interactive installation. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 2213--2222. Google ScholarDigital Library
- Ardito, C., Buono, P., Costabile, M. F., and Desolda, G. Interaction with large displays: A survey. ACM Computing Surveys (CSUR) 47, 3 (2015), 46. Google ScholarDigital Library
- Baudel, T., and Beaudouin-Lafon, M. Charade: remote control of objects using free-hand gestures. Communications of the ACM (1993). Google ScholarDigital Library
- Brignull, H., and Rogers, Y. Enticing people to interact with large public displays in public spaces. In Proceedings of INTERACT, vol. 3 (2003), 17--24.Google Scholar
- Carrino, S., Caon, M., Khaled, O. A., Ingold, R., and Mugellini, E. Functional gestures for human-environment interaction. In Human-Computer Interaction. Interaction Modalities and Techniques. Springer, 2013, 167--176. Google ScholarDigital Library
- Chattopadhyay, D., and Bolchini, D. Motor-intuitive interactions based on image schemas: Aligning touchless interaction primitives with human sensorimotor abilities. Interacting with Computers (2015).Google Scholar
- Dalton, N., Collins, E., and Marshall, P. Display blindness? looking again at the visibility of situated displays using eye tracking. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2015). Google ScholarDigital Library
- Davies, N., Clinch, S., and Alt, F. Pervasive displays: understanding the future of digital signage. Synthesis Lectures on Mobile and Pervasive Computing 8 (2014), 1--128. Google ScholarDigital Library
- Grace, K., Wasinger, R., Ackad, C., Collins, A., Dawson, O., Gluga, R., Kay, J., and Tomitsch, M. Conveying Interactivity at an Interactive Public Information Display. In the 2nd ACM International Symposium on Pervasive Displays, ACM (2013), 19--24. Google ScholarDigital Library
- Hardy, J., Rukzio, E., and Davies, N. Real world responses to interactive gesture based public displays. In the 10th International Conference on Mobile and Ubiquitous Multimedia, ACM (2011), 33--39. Google ScholarDigital Library
- Hespanhol, L., Tomitsch, M., Grace, K., Collins, A., and Kay, J. Investigating intuitiveness and effectiveness of gestures for free spatial interaction with large displays. In the 2012 International Symposium on Pervasive Displays - PerDis '12, ACM Press (2012), 1--6. Google ScholarDigital Library
- Hincapié-Ramos, J. D., Guo, X., Moghadasian, P., and Irani, P. Consumed endurance: A metric to quantify arm fatigue of mid-air interactions. In the 32nd annual ACM conference on Human factors in computing systems, ACM (2014), 1063--1072. Google ScholarDigital Library
- Houben, S., and Weichel, C. Overcoming interaction blindness through curiosity objects. In CHI '13 Extended Abstracts on Human Factors in Computing Systems, ACM (2013), 1539--1544. Google ScholarDigital Library
- Huang, E. M., Koster, A., and Borchers, J. Overcoming Assumptions and Uncovering Practices: When Does the Public Really Look at Public Displays? In the 6th International Conference on Pervasive Computing (Pervasive), Springer-Verlag (2008), 228--243. Google ScholarDigital Library
- Kukka, H., Oja, H., Kostakos, V., Goncalves, J., and Ojala, T. What makes you click: exploring visual signals to entice interaction on public displays. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 1699--1708. Google ScholarDigital Library
- Lee, L., Javed, Y., Danilowicz, S., and Maher, M. L. Information at the wave of your hand. In Proceedings of HCI Korea, HCIK '15, Hanbit Media, Inc. (2014), 63--70. Google ScholarDigital Library
- Mäkelä, V., Heimonen, T., Luhtala, M., and Turunen, M. Information wall: evaluation of a gesture-controlled public display. In the 13th International Conference on Mobile and Ubiquitous Multimedia, ACM (2014), 228--231. Google ScholarDigital Library
- Mäkelä, V., Heimonen, T., and Turunen, M. Magnetic cursor: Improving target selection in freehand pointing interfaces. In the International Symposium on Pervasive Displays, PerDis '14, ACM (2014), 112--117. Google ScholarDigital Library
- Memarovic, N., Langheinrich, M., Cheverst, K., Taylor, N., and Alt, F. P-layers--a layered framework addressing the multifaceted issues facing community-supporting public display deployments. ACM Transactions on Computer-Human Interaction (TOCHI) 20, 3 (2013), 17. Google ScholarDigital Library
- Morris, M. R., Wobbrock, J. O., and Wilson, A. D. Understanding Users' Preferences for Surface Gestures. In Proceedings of Graphics Interface 2010, GI '10, Canadian Information Processing Society (2010), 261--268. Google ScholarDigital Library
- Müller, J., Alt, F., Michelis, D., and Schmidt, A. Requirements and Design Space for Interactive Public Displays. In the International Conference on Multimedia (MM), ACM (2010), 1285--1294. Google ScholarDigital Library
- Müller, J., Walter, R., Bailly, G., Nischt, M., and Alt, F. Looking Glass: A Field Study on Noticing Interactivity of a Shop Window. In the SIGCHI Conference on Human Factors in Computing Systems (CHI), ACM (2012), 297--306. Google ScholarDigital Library
- Müller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt, A., Jay, T., and Krüger, A. Display Blindness: The Effect of Expectations on Attention towards Digital Signage. In the 7th International Conference on Pervasive Computing (Pervasive), Springer-Verlag (2009), 1--8. Google ScholarDigital Library
- Nacenta, M. A., Kamber, Y., Qiang, Y., and Kristensson, P. O. Memorability of pre-designed and user-defined gesture sets. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 1099--1108. Google ScholarDigital Library
- Nancel, M., Wagner, J., Pietriga, E., Chapuis, O., and Mackay, W. Mid-air pan-and-zoom on wall-sized displays. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2011), 177--186. Google ScholarDigital Library
- Nielsen, M., Störring, M., Moeslund, T. B., and Granum, E. A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In Gesture-Based Communication in Human-Computer Interaction. Springer, 2004, 409--420.Google Scholar
- Oh, U., and Findlater, L. The challenges and potential of end-user gesture customization. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 1129--1138. Google ScholarDigital Library
- Ojala, T., Kostakos, V., Kukka, H., Heikkinen, T., Linden, T., Jurmu, M., Hosio, S., Kruger, F., and Zanni, D. Multipurpose interactive public displays in the wild: Three years later. Computer 45, 5 (2012), 42--49. Google ScholarDigital Library
- Parra, G., Klerkx, J., and Duval, E. Understanding engagement with interactive public displays: an awareness campaign in the wild. In the International Symposium on Pervasive Displays, ACM (2014), 180. Google ScholarDigital Library
- Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., Oulasvirta, A., and Saarikko, P. It's Mine, Don't Touch!: interactions at a large multi-touch display in a city centre. In CHI '08, ACM Press (2008), 1285. Google ScholarDigital Library
- Quek, F., McNeill, D., Bryll, R., Duncan, S., Ma, X.-F., Kirbas, C., McCullough, K. E., and Ansari, R. Multimodal human discourse: gesture and speech. ACM Transactions on Computer-Human Interaction (TOCHI) 9 (2002), 171--193. Google ScholarDigital Library
- Rico, J., and Brewster, S. Usable gestures for mobile interfaces: evaluating social acceptability. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2010), 887--896. Google ScholarDigital Library
- Steinberger, F., Foth, M., and Alt, F. Vote with your feet: Local community polling on urban screens. In the International Symposium on Pervasive Displays, ACM (2014), 44. Google ScholarDigital Library
- Taylor, N., Marshall, J., Blum-Ross, A., Mills, J., Rogers, J., Egglestone, P., Frohlich, D. M., Wright, P., and Olivier, P. Viewpoint: empowering communities with situated voting devices. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2012), 1361--1370. Google ScholarDigital Library
- Ten Koppel, M., Bailly, G., Müller, J., and Walter, R. Chained displays: Configurations of Public Displays Can Be Used to Influence Actor-, Audience-, and Passer-By Behavior. In the 2012 ACM annual conference on Human Factors in Computing Systems - CHI '12, CHI '12, ACM Press (2012), 317. Google ScholarDigital Library
- Tomitsch, M., Ackad, C., Dawson, O., Hespanhol, L., and Kay, J. Who cares about the content? an analysis of playful behaviour at a public display. In the International Symposium on Pervasive Displays, ACM (2014), 160. Google ScholarDigital Library
- Valkanova, N., Jorda, S., Tomitsch, M., and Vande Moere, A. Reveal-it!: the impact of a social visualization projection on public awareness and discourse. In the SIGCHI Conference on Human Factors in Computing Systems, ACM (2013), 3461--3470. Google ScholarDigital Library
- Valkanova, N., Walter, R., Vande Moere, A., and Müller, J. Myposition: Sparking civic discourse by a public interactive poll visualization. In the 17th ACM conference on Computer supported cooperative work & social computing, ACM (2014), 1323--1332. Google ScholarDigital Library
- Vanoni, D. Human-Centered Interfaces for Large, High-Resolution Visualization Systems. In Research Exam, September 2013 (UC San Diego Computer Science and Engineering Department, retrieved May, 2014).Google Scholar
- Vogel, D., and Balakrishnan, R. Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In the 17th annual ACM symposium on User interface software and technology, ACM (2004), 137--146. Google ScholarDigital Library
- Vogel, D., and Balakrishnan, R. Distant freehand pointing and clicking on very large, high resolution displays. In the 18th annual ACM symposium on User interface software and technology, ACM (2005), 33--42. Google ScholarDigital Library
- Walter, R., Bailly, G., and Müller, J. StrikeAPose: Revealing Mid-air Gestures on Public Displays. In the SIGCHI Conference on Human Factors in Computing Systems, CHI '13, ACM (2013), 841--850. Google ScholarDigital Library
- Walter, R., Bailly, G., Valkanova, N., and Müller, J. Cuenesics: using mid-air gestures to select items on interactive public displays. In the 16th international conference on Human-computer interaction with mobile devices & services, ACM (2014), 299--308. Google ScholarDigital Library
Index Terms
- An in-the-wild study of learning mid-air gestures to browse hierarchical information at a large interactive public display
Recommendations
Eliciting Mid-Air Gestures for Wall-Display Interaction
NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer InteractionFreehand mid-air gestures are a promising input method for interacting with wall displays. However, work on mid-air gestures for wall-display interaction has mainly explored what is technically possible, which might not result in gestures that users ...
Measuring interactivity at an interactive public information display
OzCHI '13: Proceedings of the 25th Australian Computer-Human Interaction Conference: Augmentation, Application, Innovation, CollaborationPublic Information Displays (PIDs) have only recently begun to support user interaction. Traditionally, such displays have been static and non-interactive, and past research has shown that users of such displays (both non-interactive and interactive) ...
Bimanual Word Gesture Keyboards for Mid-air Gestures
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing SystemsMid-air hand gestural interaction has generally been researched as a pointing device. However, recent research has shown potential for text input with the use of word gesture keyboards (WGK), where these forms of interactions require the input system to ...
Comments