Chloe Ng
Nicolai Marquardt
Skip Abstract Section
Abstract
Many interaction techniques have been developed to best support mobile gaming – but developed gestures and techniques might not always match user behaviour or preferences. To inform this design space of gesture input for co-located mobile gaming, we present insights from a gesture elicitation user study for touch and mid-air input, specifically focusing on board and card games due to the materiality of game artefacts and rich interaction between players. We obtained touch and mid-air gesture proposals for 11 game tasks with 12 dyads and gained insights into user preferences. We contribute our classification and analysis of 622 elicited gestures (showing more collaborative gestures in the mid-air modality), resulting in a consensus gesture set, and agreement rates showing higher consensus for touch gestures. Furthermore, we identified interaction patterns – such as benefits of situational awareness, social etiquette, gestures fostering interaction between players, and roles of gestures providing fun, excitement, and suspense to the game – which can inform future games and gesture design.
- Roland Aigner, Daniel Wigdor, Hrvoje Benko, Michael Haller, David Lindbauer, Alexandra Ion, Shengdong Zhao, and Jeffrey Tzu Kwan Valino Koh. 2012. Understanding Mid-Air Hand Gestures: A Study of Human Preferences in Usage of Gesture Types for HCI. MSR-TR-2012-111 Technical Report. Retrieved May 23, 2022 from https://www.microsoft.com/en-us/research/publication/understandingmid-air-hand-gestures-a-study-of-human-preferences-in-usage-of-gesture-types-for-hci/
Google Scholar
- Alexandre Almeida and Ana Alves. 2017. Activity recognition for movement-based interaction in mobile games. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17), 1-8. https://doi.org/10.1145/3098279.3125443
Google ScholarDigital Library
- Saskia Bakker, Debby Vorstenbosch, Elise van den Hoven, Gerard Hollemans, and Tom Bergman. 2007. Weathergods: Tangible Interaction in a Digital Tabletop Game. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07), 151-152. https://doi.org/10.1145/1226969.1227000
Google ScholarDigital Library
- Rob van Bekkum, Thijs L.M. Brands, Soheil S. Jahanshahi, Aidan C.A. Mauricio, Joost J.E. Oorschot van, Fanny Lie, Ben Kybartas, and Rafael Bidarra. 2014. Taxi trouble: communication is key. In Proceedings of the first ACM SIGCHI annual symposium on Computer-human interaction in play (CHI PLAY '14), 387-390. https://doi.org/10.1145/2658537.2662994
Google ScholarDigital Library
- Po Bhattacharyya, Radha Nath, Yein Jo, Ketki Jadhav, and Jessica Hammer. 2019. Brick: Toward A Model for Designing Synchronous Colocated Augmented Reality Games. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19), 323 : 1-323 :9. https://doi.org/10.1145/3290605.3300553
Google ScholarDigital Library
- Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative Research in Psychology 3, 2 : 77-101. https://doi.org/10.1191/1478088706qp063oa
Google ScholarCross Ref
- Philipp Brauner, Julia van Heek, Martina Ziefle, Nur Al-huda Hamdan, and Jan Borchers. 2017. Interactive FUrniTURE: Evaluation of Smart Interactive Textile Interfaces for Home Environments. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17), 151-160. https://doi.org/10.1145/3132272.3134128
Google ScholarDigital Library
- Liselott Brunnberg. 2004. The Road Rager: making use of traffic encounters in a mobile multiplayer game. In Proceedings of the 3rd international conference on Mobile and ubiquitous multimedia (MUM '04), 33-39. https://doi.org/10.1145/1052380.1052386
Google ScholarDigital Library
- Edwin Chan, Teddy Seyed, Wolfgang Stuerzlinger, Xing-Dong Yang, and Frank Maurer. 2016. User Elicitation on Single-hand Microgestures. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 3403-3414. https://doi.org/10.1145/2858036.2858589
Google ScholarDigital Library
- Chi Tai Dang and Elisabeth André. 2010. Surface-poker: Multimodality in Tabletop Games. In Extended Abstracts of ACM International Conference on Interactive Tabletops and Surfaces (ITS '10), 251-252. https://doi.org/10.1145/1936652.1936701
Google ScholarDigital Library
- Giuseppe Desolda, Carmelo Ardito, Hans-Christian Jetter, and Rosa Lanzilotti. 2019. Exploring spatially-aware cross-device interaction techniques for mobile collaborative sensemaking. International Journal of Human-Computer Studies 122 : 1-20. https://doi.org/10.1016/j.ijhcs. 2018. 08.006
Google ScholarCross Ref
- Tilman Dingler, Rufat Rzayev, Alireza Sahami Shirazi, and Niels Henze. 2018. Designing Consistent Gestures Across Device Types: Eliciting RSVP Controls for Phone, Watch, and Glasses. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), 419 : 1-419 : 12. https://doi.org/10.1145/3173574.3173993
Google ScholarDigital Library
- Guiying Du, Auriol Degbelo, Christian Kray, and M. Painho. 2018. Gestural interaction with 3D objects shown on public displays: an elicitation study. Interaction Design and Architecture(s) 38 : 184-202. https://doi.org/10.55612/s-5002-038-010
Google ScholarCross Ref
- Megan Ellis. 2017. 5 Mobile Games You Control in Crazy Ways. Retrieved May 23, 2022-from https://www.makeuseof.com/tag/mobile-games-crazy-control/.
Google Scholar
- Leila Homaeian, Nippun Goyal, James R. Wallace, and Stacey D. Scott. 2018. Group vs Individual: Impact of TOUCH and TILT Cross-Device Interactions on Mixed-Focus Collaboration. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), 73 : 1-73 : 13. https://doi.org/10.1145/3173574.3173647
Google ScholarDigital Library
- itch.io. 2022. Top games for Android with Accelerometer support. itch.io. Retrieved May 23, 2022-https://itch.io/games/input-accelerometer/platform-android.
Google Scholar
- Ahmed S. Khalaf, Sultan A. Alharthi, Igor Dolgov, and Z O. Toups. 2019. A Comparative Study of Hand Gesture Recognition Devices in the Context of Game Design. In Proceedings of the 2019 ACM International Conference on Interactive Surfaces and Spaces (ISS '19), 397-402. https://doi.org/10.1145/3343055.3360758
Google ScholarDigital Library
- Seungki Kim, Donghyeon Ko, and Woohun Lee. 2017. Utilizing Smartphones as a Multi-Device Single Display Groupware to Design Collaborative Games. In Proceedings of the 2017 Conference on Designing Interactive Systems-DIS '17, 1341-1352. https://doi.org/10.1145/3064663.3064716
Google ScholarDigital Library
- Christian Kray, Daniel Nesbitt, John Dawson, and Michael Rohs. 2010. User-defined Gestures for Connecting Mobile Phones, Public Displays, and Tabletops. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '10), 239-248. https://doi.org/10.1145/1851600.1851640
Google ScholarDigital Library
- Janne Lautamäki and Riku Suomela. 2008. Using player proximity in mobile multiplayer games: experiences from Sandman. In Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology (ACE '08), 248-251. https://doi.org/10.1145/1501750.1501807
Google ScholarDigital Library
- Oleksandr Lobunets and Wolfgang Prinz. 2011. Evaluating a smart working environment with a digital card game prototype. In Proceedings of the ACM 2011 conference on Computer supported cooperative work (CSCW '11), 673-676. https://doi.org/10.1145/1958824.1958942
Google ScholarDigital Library
- Tony Manninen. 2004. Interaction Forms and Communicative Actions in Multiplayer Games. Retrieved May 23, 2022-from http://www.gamestudies.org/0301/manninen/.
Google Scholar
- Keenan R. May, Thomas M. Gable, and Bruce N. Walker. 2017. Designing an In-Vehicle Air Gesture Set Using Elicitation Methods. In Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '17), 74-83. https://doi.org/10.1145/3122986.3123015
Google ScholarDigital Library
- Erin McAweeney, Haihua Zhang, and Michael Nebeling. 2018. User-Driven Design Principles for Gesture Representations. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18), 1-13. https://doi.org/10.1145/3173574.3174121
Google ScholarDigital Library
- Meredith Ringel Morris. 2012. Web on the Wall: Insights from a Multimodal Interaction Elicitation Study. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (ITS '12), 95-104. https://doi.org/10.1145/2396636.2396651
Google ScholarDigital Library
- Meredith Ringel Morris, Andreea Danielescu, Steven Drucker, Danyel Fisher, Bongshin Lee, c. schraefel, and Jacob O. Wobbrock. 2014. Reducing legacy bias in gesture elicitation studies. interactions 21, 3: 40-45. https://doi.org/10.1145/2591689
Google ScholarDigital Library
- Meredith Ringel Morris, Anqi Huang, Andreas Paepcke, and Terry Winograd. 2006. Cooperative gestures: multi-user gestural interactions for co-located groupware. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06), 1201-1210. https://doi.org/10.1145/1124772.1124952
Google ScholarDigital Library
- Michael Nielsen, Moritz Stoerring, Thomas Moeslund, and Erik Granum. 2003. A Procedure for Developing Intuitive and Ergonomic Gesture Interfaces for HCI. 409-420. https://doi.org/10.1007/978-3-540-24598-8_38
Google ScholarCross Ref
- Steve Orzel. 2015. In the Lab: Designing Myo. Steve Orzel discusses the industrial design of the Myo armband. Retrieved May 23, 2022-from https://developerblog.myo.com/myo-industrial-design/.
Google Scholar
- Thomas Plank, Hans-Christian Jetter, Roman Rädle, Clemens N. Klokmose, Thomas Luger, and Harald Reiterer. 2017. Is Two Enough? ! Studying Benefits, Barriers, and Biases of Multi-Tablet Use for Collaborative Visualization. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17), 4548-4560. https://doi.org/10.1145/3025453.3025537
Google ScholarDigital Library
- Roman Rädle, Hans-Christian Jetter, Mario Schreiner, Zhihao Lu, Harald Reiterer, and Yvonne Rogers. 2015. Spatially-aware or Spatially-agnostic? Elicitation and Evaluation of User-Defined Cross-Device
Google Scholar
- Isabel Benavente Rodriguez and Nicolai Marquardt. 2017. Gesture Elicitation Study on How to Opt-in & Opt-out from Interactions with Public Displays. In Proceedings of the 2017 ACM International Conference on Interactive Surfaces and Spaces (ISS '17), 32-41. https://doi.org/10.1145/3132272.3134118
Google ScholarDigital Library
- Melissa J. Rogerson, Martin Gibbs, and Wally Smith. 2016. “I Love All the Bits”: The Materiality of Boardgames. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 3956-3969. https://doi.org/10.1145/2858036.2858433
Google ScholarDigital Library
- Melissa j. Rogerson, Lucy A. Sparrow, and Martin R. Gibbs. 2021. Unpacking “Boardgames with Apps”: The Hybrid Digital Boardgame Model. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI '21), 1-17. https://doi.org/10.1145/3411764.3445077
Google ScholarDigital Library
- Jaime Ruiz, Yang Li, and Edward Lank. 2011. User-defined motion gestures for mobile interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 197-206. https://doi.org/10.1145/1978942.1978971
Google ScholarDigital Library
- Douglas Schuler and Aki Namioka (eds.). 1993. Participatory Design: Principles and Practices. CRC Press, Hillsdale, N.J.
Google ScholarDigital Library
- Teddy Seyed, Chris Burns, Mario Costa Sousa, Frank Maurer, and Anthony Tang. 2012. Eliciting Usable Gestures for Multi-display Environments. In Proceedings of the 2012 ACM International Conference on Interactive Tabletops and Surfaces (ITS '12), 41-50. https://doi.org/10.1145/2396636.2396643
Google ScholarDigital Library
- Alireza Sahami Shirazi, Tanja Döring, Pouyan Parvahan, Bernd Ahrens, and Albrecht Schmidt. 2009. Poker Surface: Combining a Multi-touch Table and Mobile Phones in Interactive Card Games. In Proceedings of the 11th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '09), 73 : 1-73 :2. https://doi.org/10.1145/1613858.1613945
Google ScholarDigital Library
- Chaklam Silpasuwanchai and Xiangshi Ren. 2015. Designing concurrent full-body gestures for intense gameplay. International Journal of Human-Computer Studies 80 : 1-13. https://doi.org/10.1016/j.ijhcs. 2015. 02.010
Google ScholarDigital Library
- Mikael B. Skov, Jesper Kjeldskov, Jeni Paay, Heidi P. Jensen, and Marius P. Olsen. 2015. Investigating Cross-Device Interaction Techniques: A Case of Card Playing on Handhelds and Tablets. In Proceedings of the Annual Meeting of the Australian Special Interest Group for Computer Human Interaction (OzCHI '15), 446-454. https://doi.org/10.1145/2838739.2838763
Google ScholarDigital Library
- Dorothé Smit, Bernhard Maurer, Martin Murer, Jens Reinhardt, and Katrin Wolf. 2019. Be the Meeple: New Perspectives on Traditional Board Games. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '19), 695-698. https://doi.org/10.1145/3294109.3295657
Google ScholarDigital Library
- Yanke Tan, Sang Ho Yoon, and Karthik Ramani. 2017. BikeGesture: User Elicitation and Performance of Micro Hand Gesture as Input for Cycling. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17), 2147-2154. https://doi.org/10.1145/3027063.3053075
Google ScholarDigital Library
- Shan-Yuan Teng, Mu-Hsuan Chen, and Yung-Ta Lin. 2017. Way Out: A Multi-Layer Panorama Mobile Game Using Around-Body Interactions. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17), 230-233. https://doi.org/10.1145/3027063.3048410
Google ScholarDigital Library
- Ying-Chao Tung, Chun-Yen Hsu, Han-Yu Wang, Silvia Chyou, Jhe-Wei Lin, Pei-Jung Wu, Andries Valstar, and Mike Y. Chen. 2015. User-Defined Game Input for Smart Glasses in Public Space. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 3327-3336. https://doi.org/10.1145/2702123.2702214
Google ScholarDigital Library
- Consuelo Valdes, Diana Eastman, Casey Grote, Shantanu Thatte, Orit Shaer, Ali Mazalek, Brygg Ullmer, and Miriam K. Konkel. 2014. Exploring the Design Space of Gestural Interaction with Active Tokens Through User-defined Gestures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 4107-4116. https://doi.org/10.1145/2556288.2557373
Google ScholarDigital Library
- Radu-Daniel Vatavu and Jacob O. Wobbrock. 2015. Formalizing Agreement Analysis for Elicitation Studies: New Measures, Significance Test, and Toolkit. In Proceedings of the 33rd Annual ACM Conference on Human Factors https://doi.org/10.1145/2702123.2702223
Google ScholarDigital Library
- Panagiotis Vogiatzidakis and Panayiotis Koutsabasis. 2018. Gesture Elicitation Studies for Mid-Air Interaction: A Review. Multimodal Technologies and Interaction 2, 4 : 65. https://doi.org/10.3390/mti2040065
Google ScholarCross Ref
- Panagiotis Vogiatzidakis and Panayiotis Koutsabasis. 2019. Frame-Based Elicitation of Mid-Air Gestures for a Smart Home Device Ecosystem. Informatics 6, 2 : 23. https://doi.org/10.3390/informatics6020023
Google ScholarCross Ref
- Jacob O. Wobbrock, Htet Htet Aung, Brandon Rothrock, and Brad A. Myers. 2005. Maximizing the guessability of symbolic input. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05), 1869-1872. https://doi.org/10.1145/1056808.1057043
Google ScholarDigital Library
- Jacob O. Wobbrock, Meredith Ringel Morris, and Andrew D. Wilson. 2009. User-defined Gestures for Surface Computing. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 1083-1092. https://doi.org/10.1145/1518701.1518866
Google ScholarDigital Library
- Taoshuai Zhang, Jie Liu, and Yuanchun Shi. 2012. Enhancing Collaboration in Tabletop Board Game. In Proceedings of the 10th Asia Pacific Conference on Computer Human Interaction (APCHI '12), 7-10. https://doi.org/10.1145/2350046.2350050
Google ScholarDigital Library
Index Terms
- Eliciting User-Defined Touch and Mid-air Gestures for Co-located Mobile Gaming
Recommendations
Eliciting Mid-Air Gestures for Wall-Display Interaction
NordiCHI '16: Proceedings of the 9th Nordic Conference on Human-Computer InteractionFreehand mid-air gestures are a promising input method for interacting with wall displays. However, work on mid-air gestures for wall-display interaction has mainly explored what is technically possible, which might not result in gestures that users ...
Pen + Mid-Air Gestures: Eliciting Contextual Gestures
ICMI '18: Proceedings of the 20th ACM International Conference on Multimodal InteractionCombining mid-air gestures with pen input for bi-manual input on tablets has been reported as an alternative and attractive input technique in drawing applications. Previous work has also argued that mid-air gestural input can cause discomfort and arm ...
User-defined Interface Gestures: Dataset and Analysis
ITS '14: Proceedings of the Ninth ACM International Conference on Interactive Tabletops and SurfacesWe present a video-based gesture dataset and a methodology for annotating video-based gesture datasets. Our dataset consists of user-defined gestures generated by 18 participants from a previous investigation of gesture memorability. We design and use a ...
Comments