skip to main content
10.1145/3491102.3517555acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

Machine-Mediated Teaming: Mixture of Human and Machine in Physical Gaming Experience

Published: 29 April 2022 Publication History

Abstract

Technological advancement has opened up opportunities for new sports and physical activities. We introduce a concept called machine-mediated teaming, in which a human and a surrogate machine form a team to participate in physical sports games. To understand the experience of machine-mediated teaming and the guidelines for designing the system to achieve the concept, we built a case study system based on tug-of-war. Our system is a sports game played by two against two. One team consists of a player who actually pulls the rope and another player who participates in the physical game by controlling the machine’s actuators. We conducted user studies using this system to investigate the sport experience in this form and to reveal insights to inform future research on machine-mediated teaming. Based on the data obtained from the user studies, we clarified three perspectives, machine stamina, action space, and explicit feedback, that should be considered when designing future machine-mediated teaming systems. The research presented in this paper offers a first step towards exploring how humans and machines can coexist in highly dynamic physical interactions.

Supplemental Material

MP4 File
Video Figure

References

[1]
Gareth Barnaby and Anne Roudaut. 2019. Mantis: A Scalable, Lightweight and Accessible Architecture to Build Multiform Force Feedback Systems. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 937–948. https://doi.org/10.1145/3332165.3347909
[2]
Patrick Baudisch, Henning Pohl, Stefanie Reinicke, Emilia Wittmers, Patrick Lühne, Marius Knaust, Sven Köhler, Patrick Schmidt, and Christian Holz. 2013. Imaginary reality gaming: ball games without a ball. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 405–410.
[3]
William Benter. 2008. Computer based horse race handicapping and wagering systems: a report. In Efficiency of racetrack betting markets. World Scientific, 183–198.
[4]
Nadia Bianchi-Berthouze, Whan Woong Kim, and Darshak Patel. 2007. Does body movement engage you more in digital game play? and why?. In International conference on affective computing and intelligent interaction. Springer, 102–113.
[5]
Andrea Calvo, Adriano Chiò, Emiliano Castellina, Fulvio Corno, Laura Farinetti, Paolo Ghiglione, Valentina Pasian, and Alessandro Vignola. 2008. Eye tracking impact on quality-of-life of ALS patients. In International Conference on Computers for Handicapped Persons. Springer, 70–77.
[6]
Adrian David Cheok, Kok Hwee Goh, Wei Liu, Farzam Farbiz, Siew Wan Fong, Sze Lee Teo, Yu Li, and Xubo Yang. 2004. Human Pacman: a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing. Personal and ubiquitous computing 8, 2 (2004), 71–81.
[7]
Martin Feick, Terrance Mok, Anthony Tang, Lora Oehlberg, and Ehud Sharlin. 2018. Perspective on and Re-Orientation of Physical Proxies in Object-Focused Remote Collaboration. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173855
[8]
Kathrin Gerling, Ian Livingston, Lennart Nacke, and Regan Mandryk. 2012. Full-body motion-based game interaction for older adults. In Proceedings of the SIGCHI conference on human factors in computing systems. 1873–1882.
[9]
Barney G Glaser and Anselm L Strauss. 2017. Discovery of Grounded Theory: Strategies for Qualitative Research. Routledge. 282 pages.
[10]
Lee Graves, Gareth Stratton, Nicola D Ridgers, and N Tim Cable. 2007. Comparison of energy expenditure in adolescents when playing new generation and sedentary computer games: cross sectional study. Bmj 335, 7633 (2007), 1282–1284.
[11]
Hiroshi Ishii, Craig Wisneski, Julian Orbanes, Ben Chun, and Joe Paradiso. 1999. PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 394–401.
[12]
Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016. Laplacian vision: Augmenting motion prediction via optical see-through head-mounted displays. In Proceedings of the 7th Augmented Human International Conference 2016. 1–8.
[13]
David Kirschner, Andreas Schlotzhauer, Mathias Brandstötter, and Michael Hofbaur. 2017. Validation of relevant parameters of sensitive manipulators for human-robot collaboration. In International Conference on Robotics in Alpe-Adria Danube Region. Springer, 242–252.
[14]
Kai Kunze, Kouta Minamizawa, Stephan Lukosch, Masahiko Inami, and Jun Rekimoto. 2017. Superhuman sports: Applying human augmentation to physical exercise. IEEE Pervasive Computing 16, 2 (2017), 14–17.
[15]
Lorraine Lanningham-Foster, Teresa B Jensen, Randal C Foster, Aoife B Redmond, Brian A Walker, Dieter Heinz, and James A Levine. 2006. Energy expenditure of sedentary screen time compared with active screen time for children. Pediatrics 118, 6 (2006), e1831–e1835.
[16]
Jong-Hwan Lee, Jeongwon Ryu, Ferenc A Jolesz, Zang-Hee Cho, and Seung-Schik Yoo. 2009. Brain–machine interface via real-time fMRI: preliminary study on thought-controlled robotic arm. Neuroscience letters 450, 1 (2009), 1–6.
[17]
Min Kyung Lee and Leila Takayama. 2011. ”Now, i Have a Body”: Uses and Social Norms for Mobile Remote Presence in the Workplace. Association for Computing Machinery, New York, NY, USA, 33–42. https://doi.org/10.1145/1978942.1978950
[18]
Daniel Leithinger, Sean Follmer, Alex Olwal, and Hiroshi Ishii. 2014. Physical Telepresence: Shape Capture and Display for Embodied, Computer-Mediated Remote Collaboration(UIST ’14). Association for Computing Machinery, New York, NY, USA, 461–470. https://doi.org/10.1145/2642918.2647377
[19]
Siân E Lindley, James Le Couteur, and Nadia L Berthouze. 2008. Stirring up experience through movement in game play: effects on engagement and social behaviour. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 511–514.
[20]
Lian Loke, Astrid T Larssen, Toni Robertson, and Jenny Edwards. 2007. Understanding movement for interaction design: frameworks and approaches. Personal and Ubiquitous Computing 11, 8 (2007), 691–701.
[21]
Azumi Maekawa, Seito Matsubara, Sohei Wakisaka, Daisuke Uriu, Atsushi Hiyama, and Masahiko Inami. 2020. Dynamic Motor Skill Synthesis with Human-Machine Mutual Actuation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). ACM, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376705
[22]
Elena Márquez Segura, Annika Waern, Jin Moen, and Carolina Johansson. 2013. The design space of body games: technological, physical, and social design. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 3365–3374.
[23]
Joe Marshall, Steve Benford, Sebastiaan Pijnappel, 2016. Expanding exertion gaming. International Journal of Human-Computer Studies 90 (2016), 1–13.
[24]
Wendy Moyle, Urska Arnautovska, Tamara Ownsworth, and Cindy Jones. 2017. Potential of telepresence robots to enhance social connectedness in older adults with dementia: an integrative review of feasibility. International psychogeriatrics 29, 12 (2017), 1951–1964.
[25]
Florian Mueller, Stefan Agamanolis, and Rosalind Picard. 2003. Exertion interfaces: sports over a distance for social bonding and fun. In Proceedings of the SIGCHI conference on Human factors in computing systems. 561–568.
[26]
Florian’Floyd’ Mueller, Richard Byrne, Josh Andres, and Rakesh Patibanda. 2018. Experiencing the body as play. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.
[27]
Florian Mueller and Katherine Isbister. 2014. Movement-based game guidelines. In Proceedings of the sigchi conference on human factors in computing systems. 2191–2200.
[28]
Florian ’Floyd’ Mueller, Tuomas Kari, Zhuying Li, Yan Wang, Yash Dhanpal Mehta, Josh Andres, Jonathan Marquez, and Rakesh Patibanda. 2020. Towards Designing Bodily Integrated Play. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (Sydney NSW, Australia) (TEI ’20). Association for Computing Machinery, New York, NY, USA, 207–218. https://doi.org/10.1145/3374920.3374931
[29]
Florian ’Floyd’ Mueller and Damon Young. 2017. Five Lenses for Designing Exertion Experiences. Association for Computing Machinery, New York, NY, USA, 2473–2487. https://doi.org/10.1145/3025453.3025746
[30]
Florian ‘Floyd’ Mueller, Martin R Gibbs, Frank Vetere, and Darren Edge. 2017. Designing for bodily interplay in social exertion games. ACM Transactions on Computer-Human Interaction (TOCHI) 24, 3(2017), 1–41.
[31]
Jonathan Mumm and Bilge Mutlu. 2011. Human-Robot Proxemics: Physical and Psychological Distancing in Human-Robot Interaction. In Proceedings of the 6th International Conference on Human-Robot Interaction (Lausanne, Switzerland) (HRI ’11). Association for Computing Machinery, New York, NY, USA, 331–338. https://doi.org/10.1145/1957656.1957786
[32]
Carman Neustaedter, Samarth Singhal, Rui Pan, Yasamin Heshmat, Azadeh Forghani, and John Tang. 2018. From Being There to Watching: Shared and Dedicated Telepresence Robot Usage at Academic Conferences. ACM Trans. Comput.-Hum. Interact. 25, 6, Article 33 (Dec. 2018), 39 pages. https://doi.org/10.1145/3243213
[33]
Carman Neustaedter, Gina Venolia, Jason Procyk, and Daniel Hawkins. 2016. To Beam or Not to Beam: A Study of Remote Telepresence Attendance at an Academic Conference. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (San Francisco, California, USA) (CSCW ’16). Association for Computing Machinery, New York, NY, USA, 418–431. https://doi.org/10.1145/2818048.2819922
[34]
Kei Nitta, Keita Higuchi, and Jun Rekimoto. 2014. HoverBall: augmented sports with a flying ball. In Proceedings of the 5th Augmented Human International Conference. 1–4.
[35]
Kei Nitta, Keita Higuchi, Yuichi Tadokoro, and Jun Rekimoto. 2015. Shepherd pass: ability tuning for augmented sports using ball-shaped quadcopter. In Proceedings of the 12th International Conference on Advances in Computer Entertainment Technology. 1–7.
[36]
Nintendo Entertainment Planning & Development Division (EPD) Production Group No.4. 2019. Ring Fit Adventure. Retrieved August 24, 2021 from https://www.nintendo.com/products/detail/ring-fit-adventure-switch/.
[37]
Takuya Nojima, Ngoc Phuong, Takahiro Kai, Toshiki Sato, and Hideki Koike. 2015. Augmented dodgeball: an approach to designing augmented sports. In Proceedings of the 6th Augmented Human International Conference. 137–140.
[38]
Tomoya Ohta, Shumpei Yamakawa, Takashi Ichikawa, and Takuya Nojima. 2014. TAMA: development of trajectory changeable ball for future entertainment. In Proceedings of the 5th Augmented Human International Conference. 1–8.
[39]
Hannah R. M. Pelikan, Amy Cheatle, Malte F. Jung, and Steven J. Jackson. 2018. Operating at a Distance - How a Teleoperated Surgical Robot Reconfigures Teamwork in the Operating Room. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 138 (Nov. 2018), 28 pages. https://doi.org/10.1145/3274407
[40]
Irene Rae and Carman Neustaedter. 2017. Robotic Telepresence at Scale. Association for Computing Machinery, New York, NY, USA, 313–324. https://doi.org/10.1145/3025453.3025855
[41]
Alec Rivers, Ilan E. Moyer, and Frédo Durand. 2012. Position-Correcting Tools for 2D Digital Fabrication. ACM Trans. Graph. 31, 4, Article 88 (July 2012), 7 pages. https://doi.org/10.1145/2185520.2185584
[42]
MHD Yamen Saraiji, Tomoya Sasaki, Reo Matsumura, Kouta Minamizawa, and Masahiko Inami. 2018. Fusion: full body surrogacy for collaborative communication. In ACM SIGGRAPH 2018 Emerging Technologies. 1–2.
[43]
Koya Sato, Yuji Sano, Mai Otsuki, Mizuki Oka, and Kazuhiko Kato. 2019. Augmented Recreational Volleyball Court: Supporting the Beginners’ Landing Position Prediction Skill by Providing Peripheral Visual Feedback. In Proceedings of the 10th Augmented Human International Conference 2019. 1–9.
[44]
Roy Shilkrot, Pattie Maes, Joseph A. Paradiso, and Amit Zoran. 2015. Augmented Airbrush for Computer Aided Painting (CAP). ACM Trans. Graph. 34, 2, Article 19 (March 2015), 11 pages. https://doi.org/10.1145/2699649
[45]
Brett Stoll, Samantha Reig, Lucy He, Ian Kaplan, Malte F. Jung, and Susan R. Fussell. 2018. Wait, Can You Move the Robot? Examining Telepresence Robot Use in Collaborative Teams. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (Chicago, IL, USA) (HRI ’18). Association for Computing Machinery, New York, NY, USA, 14–22. https://doi.org/10.1145/3171221.3171243
[46]
Hang Su, Chenguang Yang, Giancarlo Ferrigno, and Elena De Momi. 2019. Improved human–robot collaborative control of redundant robot for teleoperated minimally invasive surgery. IEEE Robotics and Automation Letters 4, 2 (2019), 1447–1453.
[47]
Ryo Suzuki, Hooman Hedayati, Clement Zheng, James L. Bohn, Daniel Szafir, Ellen Yi-Luen Do, Mark D. Gross, and Daniel Leithinger. 2020. RoomShift: Room-Scale Dynamic Haptics for VR with Furniture-Moving Swarm Robots. Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376523
[48]
Tim B Swartz. 2009. A new handicapping system for golf. Journal of Quantitative Analysis in Sports 5, 2 (2009).
[49]
Yuchuang Tong and Jinguo Liu. 2021. Review of Research and Development of Supernumerary Robotic Limbs.IEEE CAA J. Autom. Sinica 8, 5 (2021), 929–952.
[50]
Jui-Hung Tu, Chien-Hsun Lee, and Yung-Hsing Chiu. 2005. The analysis of pulling force curves in Tug-of-war. In ISBS-Conference Proceedings Archive.
[51]
Gina Venolia, John Tang, Ruy Cervantes, Sara Bly, George Robertson, Bongshin Lee, and Kori Inkpen. 2010. Embodied Social Proxy: Mediating Interpersonal Connection in Hub-and-Satellite Teams. Association for Computing Machinery, New York, NY, USA, 1049–1058. https://doi.org/10.1145/1753326.1753482
[52]
Lillian Yang, Carman Neustaedter, and Thecla Schiphorst. 2017. Communicating Through A Telepresence Robot: A Study of Long Distance Relationships. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA ’17). Association for Computing Machinery, New York, NY, USA, 3027–3033. https://doi.org/10.1145/3027063.3053240
[53]
Simon Zimmermann, Roi Poranne, and Stelian Coros. 2021. Go Fetch!-Dynamic grasps using Boston Dynamics Spot with external robotic arm. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4488–4494.
[54]
Amit Zoran and Joseph A. Paradiso. 2013. FreeD: A Freehand Digital Sculpting Tool. Association for Computing Machinery, New York, NY, USA, 2613–2616. https://doi.org/10.1145/2470654.2481361

Cited By

View all
  • (2023)STAMPER: Human-machine Integrated DrummingExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585619(1-5)Online publication date: 19-Apr-2023
  • (2023)Social Digital Cyborgs: The Collaborative Design Process of JIZAI ARMSProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581169(1-19)Online publication date: 19-Apr-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems
April 2022
10459 pages
ISBN:9781450391573
DOI:10.1145/3491102
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. augmentation
  2. human-machine collaboration
  3. physical interaction
  4. sports design

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

Conference

CHI '22
Sponsor:
CHI '22: CHI Conference on Human Factors in Computing Systems
April 29 - May 5, 2022
LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)340
  • Downloads (Last 6 weeks)39
Reflects downloads up to 25 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)STAMPER: Human-machine Integrated DrummingExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585619(1-5)Online publication date: 19-Apr-2023
  • (2023)Social Digital Cyborgs: The Collaborative Design Process of JIZAI ARMSProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581169(1-19)Online publication date: 19-Apr-2023

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media