Robert Walter
Gilles Bailly
Nina Valkanova
Jörg Müller
ABSTRACT
Most of today's public displays only show predefined contents that passers-by are not able to change. We argue that interactive public displays would benefit from immediately usable mid-air techniques for choosing options, expressing opinions or more generally selecting one among several items. We propose a design space for hand-gesture based mid-air selection techniques on interactive public displays, along with four specific techniques that we evaluated at three different locations in the the field. Our findings include: 1) if no hint is provided, people successfully use Point+Dwell for selecting items, 2) the user representation could be switched from Mirror to Cursor after registration without causing confusion, 3) people tend to explore items before confirming one, 4) in a public context, people frequently interact inadvertently (without looking at the screen). We conclude by providing recommendations for designers of interactive public displays to support immediate usability for mid-air selection.
Supplemental Material
- Bailly, G., Lecolinet, E., and Guiard, Y. Finger-count; radial-stroke shortcuts: 2 techniques for augmenting linear menus on multi-touch surfaces. In CHI'10, ACM (2010), 591--594. Google Scholar
Digital Library
- Bailly, G., Vo, D.-B., Lecolinet, E., and Guiard, Y. Gesture-aware remote controls: guidelines and interaction technique. In ICMI'11, ACM (2011), 263--270. Google ScholarDigital Library
- Bailly, G., Walter, R., Müller, J., Ning, T., and Lecolinet, E. Comparing free hand menu techniques for distant displays using linear, marking and finger-count menus. In INTERACT'11, Springer (2011), 248--262. Google ScholarDigital Library
- Bowman, D. A., and Wingrave, C. A. Design and evaluation of menu systems for immersive virtual environments. In VR'01, IEEE (2001), 149--. Google ScholarDigital Library
- Brignull, H., and Rogers, Y. Enticing people to interact with large public displays in public spaces. In In Proceedings of the IFIP International Conference on Human-Computer Interaction (INTERACT 2003 (2003), 17--24.Google Scholar
- Cockburn, A., and Gin, A. Faster cascading menu selections with enlarged activation areas. In GI'06, Canadian Information Processing Society (2006), 65--71. Google ScholarDigital Library
- Francone, J., Bailly, G., Lecolinet, E., Mandran, N., and Nigay, L. Wavelet menus on handheld devices: stacking metaphor for novice mode and eyes-free selection for expert mode. In AVI'10, ACM (2010), 173--180. Google ScholarDigital Library
- Hespanhol, L., Tomitsch, M., Grace, K., Collins, A., and Kay, J. Investigating intuitiveness and effectiveness of gestures for free spatial interaction with large displays. In PerDis'12, ACM (2012), 6:1--6:6. Google ScholarDigital Library
- Huang, E. M., Koster, A., and Borchers, J. Overcoming assumptions and uncovering practices: When does the public really look at public displays? In Springer Pervasive'08 (Berlin, Heidelberg, 2008), 228--243. Google ScholarDigital Library
- Jacucci, G., Morrison, A., Richard, G. T., Kleimola, J., Peltonen, P., Parisi, L., and Laitinen, T. Worlds of information: designing for engagement at a public multi-touch display. In CHI'10, ACM (2010), 2267--2276. Google ScholarDigital Library
- Ju, W., and Sirkin, D. Animate objects: how physical motion encourages public interaction. In PERSUASIVE'10, Springer (2010), 40--51. Google ScholarDigital Library
- Kostakos, V., Kukka, H., Goncalves, J., Tselios, N., and Ojala, T. Multipurpose public displays: How shortcut menus affect usage. Computer Graphics and Applications, IEEE 33, 2 (2013), 56--63. Google ScholarDigital Library
- Krueger, M. Artificial reality II. Addison-Wesley, 1991.Google Scholar
- Kules, B., Kang, H., Plaisant, C., Rose, A., and Shneiderman, B. Immediate usability: a case study of public access design for a community photo library. Interacting with Computers 16, 6 (2004), 1171--1193.Google ScholarCross Ref
- Kurtenbach, G., and Buxton, W. User learning and performance with marking menus. In CHI'94, ACM (1994), 258--264. Google ScholarDigital Library
- Lepinski, G. J., Grossman, T., and Fitzmaurice, G. The design and evaluation of multitouch marking menus. In CHI'10, ACM (2010), 2233--2242. Google ScholarDigital Library
- Marshall, P., Morris, R., Rogers, Y., Kreitmayer, S., and Davies, M. Rethinking 'multi-user': an in-the-wild study of how groups approach a walk-up-and-use tabletop interface. In CHI'11, ACM (New York, NY, USA, 2011), 3033--3042. Google ScholarDigital Library
- Müller, J., Alt, F., Michelis, D., and Schmidt, A. Requirements and design space for interactive public displays. In MM'10, ACM (2010), 1285--1294. Google ScholarDigital Library
- Müller, J., Walter, R., Bailly, G., Nischt, M., and Alt, F. Looking glass: a field study on noticing interactivity of a shop window. In CHI'12, ACM (2012), 297--306. Google ScholarDigital Library
- Müller, J., Wilmsmann, D., Exeler, J., Buzeck, M., Schmidt, A., Jay, T., and Krüger, A. Display blindness: The effect of expectations on attention towards digital signage. In Pervasive'09, Springer (2009), 1--8. Google ScholarDigital Library
- Nilsen, E. L. Perceptual-motor control in human-computer interaction. PhD thesis, Ann Arbor, MI, USA, 1991. UMI Order No. GAX91-35659. Google ScholarDigital Library
- Peltonen, P., Kurvinen, E., Salovaara, A., Jacucci, G., Ilmonen, T., Evans, J., Oulasvirta, A., and Saarikko, P. It's mine, don't touch!: interactions at a large multi-touch display in a city centre. In CHI'08, ACM (2008), 1285--1294. Google ScholarDigital Library
- Samp, K. Designing graphical menus for novices and experts: connecting design characteristics with design goals. In CHI'13, ACM (2013), 3159--3168. Google ScholarDigital Library
- Shoemaker, G., Tsukitani, T., Kitamura, Y., and Booth, K. S. Body-centric interaction techniques for very large wall displays. In NordiCHI'10, ACM (2010), 463--472. Google ScholarDigital Library
- Song, H., Grossman, T., Fitzmaurice, G., Guimbretiere, F., Khan, A., Attar, R., and Kurtenbach, G. Penlight: combining a mobile projector and a digital pen for dynamic visual overlay. In CHI'09, ACM (2009), 143--152. Google ScholarDigital Library
- Ten Koppel, M., Bailly, G., Müller, J., and Walter, R. Chained displays: configurations of public displays can be used to influence actor-, audience-, and passer-by behavior. In CHI'12, ACM (2012), 317--326. Google ScholarDigital Library
- Valkanova, N., Walter, R., and Müller, J. Myposition: A public polling projection display designed to spark civic participation.Google Scholar
- Vogel, D., and Balakrishnan, R. Interactive public ambient displays: transitioning from implicit to explicit, public to personal, interaction with multiple users. In ACM UIST'04 (2004), 137--146. Google ScholarDigital Library
- Walter, R., Bailly, G., and Müller, J. Strikeapose: revealing mid-air gestures on public displays. In CHI'13, ACM (2013), 841--850. Google ScholarDigital Library
- Wu, M., Shen, C., Ryall, K., Forlines, C., and Balakrishnan, R. Gesture registration, relaxation, and reuse for multi-point direct-touch surfaces. In TABLETOP'06, IEEE (2006), 185--192. Google ScholarDigital Library
- Zhao, S., Dragicevic, P., Chignell, M., Balakrishnan, R., and Baudisch, P. Earpod: eyes-free menu selection using touch input and reactive audio feedback. In CHI'07, ACM (2007), 1395--1404. Google ScholarDigital Library
Index Terms
- Cuenesics: using mid-air gestures to select items on interactive public displays
Recommendations
Call to interact: communicating interactivity and affordances for contactless gesture controlled public displays
PerDis '20: Proceedings of the 9TH ACM International Symposium on Pervasive DisplaysInteractive public displays are becoming increasingly commonplace. Two important challenges for these displays exist: 1) how interactivity is communicated to the passer-by and 2) once the audience have engaged, how the interactive affordances are ...
Eliciting Mid-Air Gestures for Wall-Display Interaction
NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer InteractionFreehand mid-air gestures are a promising input method for interacting with wall displays. However, work on mid-air gestures for wall-display interaction has mainly explored what is technically possible, which might not result in gestures that users ...
Proxemic Cursor Interactions for Touchless Widget Control
SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User InteractionTouchless gesture interfaces often use cursor-based interactions, where widgets are targeted by a movable cursor and activated with a mid-air gesture (e.g., push or Pinch). Continuous interactions like slider manipulation can be challenging in mid-air ...
Comments