research-article

Public Access

Machine-Mediated Teaming: Mixture of Human and Machine in Physical Gaming Experience

Authors:

Shunichi Kasahara,

Masahiko InamiAuthors Info & Claims

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

Article No.: 618, Pages 1 - 11

https://doi.org/10.1145/3491102.3517555

Published: 29 April 2022 Publication History

All formats PDF

Abstract

Technological advancement has opened up opportunities for new sports and physical activities. We introduce a concept called machine-mediated teaming, in which a human and a surrogate machine form a team to participate in physical sports games. To understand the experience of machine-mediated teaming and the guidelines for designing the system to achieve the concept, we built a case study system based on tug-of-war. Our system is a sports game played by two against two. One team consists of a player who actually pulls the rope and another player who participates in the physical game by controlling the machine’s actuators. We conducted user studies using this system to investigate the sport experience in this form and to reveal insights to inform future research on machine-mediated teaming. Based on the data obtained from the user studies, we clarified three perspectives, machine stamina, action space, and explicit feedback, that should be considered when designing future machine-mediated teaming systems. The research presented in this paper offers a first step towards exploring how humans and machines can coexist in highly dynamic physical interactions.

Supplemental Material

MP4 File

Video Figure

Download
147.82 MB

MP4 File

Video Preview

Transcript for: Video Preview

References

[1]

Gareth Barnaby and Anne Roudaut. 2019. Mantis: A Scalable, Lightweight and Accessible Architecture to Build Multiform Force Feedback Systems. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (New Orleans, LA, USA) (UIST ’19). Association for Computing Machinery, New York, NY, USA, 937–948. https://doi.org/10.1145/3332165.3347909

Digital Library

[2]

Patrick Baudisch, Henning Pohl, Stefanie Reinicke, Emilia Wittmers, Patrick Lühne, Marius Knaust, Sven Köhler, Patrick Schmidt, and Christian Holz. 2013. Imaginary reality gaming: ball games without a ball. In Proceedings of the 26th annual ACM symposium on User interface software and technology. 405–410.

Digital Library

[3]

William Benter. 2008. Computer based horse race handicapping and wagering systems: a report. In Efficiency of racetrack betting markets. World Scientific, 183–198.

[4]

Nadia Bianchi-Berthouze, Whan Woong Kim, and Darshak Patel. 2007. Does body movement engage you more in digital game play? and why?. In International conference on affective computing and intelligent interaction. Springer, 102–113.

Digital Library

[5]

Andrea Calvo, Adriano Chiò, Emiliano Castellina, Fulvio Corno, Laura Farinetti, Paolo Ghiglione, Valentina Pasian, and Alessandro Vignola. 2008. Eye tracking impact on quality-of-life of ALS patients. In International Conference on Computers for Handicapped Persons. Springer, 70–77.

Digital Library

[6]

Adrian David Cheok, Kok Hwee Goh, Wei Liu, Farzam Farbiz, Siew Wan Fong, Sze Lee Teo, Yu Li, and Xubo Yang. 2004. Human Pacman: a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing. Personal and ubiquitous computing 8, 2 (2004), 71–81.

[7]

Martin Feick, Terrance Mok, Anthony Tang, Lora Oehlberg, and Ehud Sharlin. 2018. Perspective on and Re-Orientation of Physical Proxies in Object-Focused Remote Collaboration. Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3173574.3173855

Digital Library

[8]

Kathrin Gerling, Ian Livingston, Lennart Nacke, and Regan Mandryk. 2012. Full-body motion-based game interaction for older adults. In Proceedings of the SIGCHI conference on human factors in computing systems. 1873–1882.

Digital Library

[9]

Barney G Glaser and Anselm L Strauss. 2017. Discovery of Grounded Theory: Strategies for Qualitative Research. Routledge. 282 pages.

[10]

Lee Graves, Gareth Stratton, Nicola D Ridgers, and N Tim Cable. 2007. Comparison of energy expenditure in adolescents when playing new generation and sedentary computer games: cross sectional study. Bmj 335, 7633 (2007), 1282–1284.

[11]

Hiroshi Ishii, Craig Wisneski, Julian Orbanes, Ben Chun, and Joe Paradiso. 1999. PingPongPlus: design of an athletic-tangible interface for computer-supported cooperative play. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. 394–401.

Digital Library

[12]

Yuta Itoh, Jason Orlosky, Kiyoshi Kiyokawa, and Gudrun Klinker. 2016. Laplacian vision: Augmenting motion prediction via optical see-through head-mounted displays. In Proceedings of the 7th Augmented Human International Conference 2016. 1–8.

[13]

David Kirschner, Andreas Schlotzhauer, Mathias Brandstötter, and Michael Hofbaur. 2017. Validation of relevant parameters of sensitive manipulators for human-robot collaboration. In International Conference on Robotics in Alpe-Adria Danube Region. Springer, 242–252.

[14]

Kai Kunze, Kouta Minamizawa, Stephan Lukosch, Masahiko Inami, and Jun Rekimoto. 2017. Superhuman sports: Applying human augmentation to physical exercise. IEEE Pervasive Computing 16, 2 (2017), 14–17.

Digital Library

[15]

Lorraine Lanningham-Foster, Teresa B Jensen, Randal C Foster, Aoife B Redmond, Brian A Walker, Dieter Heinz, and James A Levine. 2006. Energy expenditure of sedentary screen time compared with active screen time for children. Pediatrics 118, 6 (2006), e1831–e1835.

[16]

Jong-Hwan Lee, Jeongwon Ryu, Ferenc A Jolesz, Zang-Hee Cho, and Seung-Schik Yoo. 2009. Brain–machine interface via real-time fMRI: preliminary study on thought-controlled robotic arm. Neuroscience letters 450, 1 (2009), 1–6.

[17]

Min Kyung Lee and Leila Takayama. 2011. ”Now, i Have a Body”: Uses and Social Norms for Mobile Remote Presence in the Workplace. Association for Computing Machinery, New York, NY, USA, 33–42. https://doi.org/10.1145/1978942.1978950

Digital Library

[18]

Daniel Leithinger, Sean Follmer, Alex Olwal, and Hiroshi Ishii. 2014. Physical Telepresence: Shape Capture and Display for Embodied, Computer-Mediated Remote Collaboration(UIST ’14). Association for Computing Machinery, New York, NY, USA, 461–470. https://doi.org/10.1145/2642918.2647377

Digital Library

[19]

Siân E Lindley, James Le Couteur, and Nadia L Berthouze. 2008. Stirring up experience through movement in game play: effects on engagement and social behaviour. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. 511–514.

Digital Library

[20]

Lian Loke, Astrid T Larssen, Toni Robertson, and Jenny Edwards. 2007. Understanding movement for interaction design: frameworks and approaches. Personal and Ubiquitous Computing 11, 8 (2007), 691–701.

Digital Library

[21]

Azumi Maekawa, Seito Matsubara, Sohei Wakisaka, Daisuke Uriu, Atsushi Hiyama, and Masahiko Inami. 2020. Dynamic Motor Skill Synthesis with Human-Machine Mutual Actuation. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). ACM, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376705

Digital Library

[22]

Elena Márquez Segura, Annika Waern, Jin Moen, and Carolina Johansson. 2013. The design space of body games: technological, physical, and social design. In Proceedings of the SIGCHI conference on Human Factors in computing systems. 3365–3374.

Digital Library

[23]

Joe Marshall, Steve Benford, Sebastiaan Pijnappel, 2016. Expanding exertion gaming. International Journal of Human-Computer Studies 90 (2016), 1–13.

Digital Library

[24]

Wendy Moyle, Urska Arnautovska, Tamara Ownsworth, and Cindy Jones. 2017. Potential of telepresence robots to enhance social connectedness in older adults with dementia: an integrative review of feasibility. International psychogeriatrics 29, 12 (2017), 1951–1964.

[25]

Florian Mueller, Stefan Agamanolis, and Rosalind Picard. 2003. Exertion interfaces: sports over a distance for social bonding and fun. In Proceedings of the SIGCHI conference on Human factors in computing systems. 561–568.

Digital Library

[26]

Florian’Floyd’ Mueller, Richard Byrne, Josh Andres, and Rakesh Patibanda. 2018. Experiencing the body as play. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13.

Digital Library

[27]

Florian Mueller and Katherine Isbister. 2014. Movement-based game guidelines. In Proceedings of the sigchi conference on human factors in computing systems. 2191–2200.

Digital Library

[28]

Florian ’Floyd’ Mueller, Tuomas Kari, Zhuying Li, Yan Wang, Yash Dhanpal Mehta, Josh Andres, Jonathan Marquez, and Rakesh Patibanda. 2020. Towards Designing Bodily Integrated Play. In Proceedings of the Fourteenth International Conference on Tangible, Embedded, and Embodied Interaction (Sydney NSW, Australia) (TEI ’20). Association for Computing Machinery, New York, NY, USA, 207–218. https://doi.org/10.1145/3374920.3374931

Digital Library

[29]

Florian ’Floyd’ Mueller and Damon Young. 2017. Five Lenses for Designing Exertion Experiences. Association for Computing Machinery, New York, NY, USA, 2473–2487. https://doi.org/10.1145/3025453.3025746

Digital Library

[30]

Florian ‘Floyd’ Mueller, Martin R Gibbs, Frank Vetere, and Darren Edge. 2017. Designing for bodily interplay in social exertion games. ACM Transactions on Computer-Human Interaction (TOCHI) 24, 3(2017), 1–41.

[31]

Jonathan Mumm and Bilge Mutlu. 2011. Human-Robot Proxemics: Physical and Psychological Distancing in Human-Robot Interaction. In Proceedings of the 6th International Conference on Human-Robot Interaction (Lausanne, Switzerland) (HRI ’11). Association for Computing Machinery, New York, NY, USA, 331–338. https://doi.org/10.1145/1957656.1957786

Digital Library

[32]

Carman Neustaedter, Samarth Singhal, Rui Pan, Yasamin Heshmat, Azadeh Forghani, and John Tang. 2018. From Being There to Watching: Shared and Dedicated Telepresence Robot Usage at Academic Conferences. ACM Trans. Comput.-Hum. Interact. 25, 6, Article 33 (Dec. 2018), 39 pages. https://doi.org/10.1145/3243213

Digital Library

[33]

Carman Neustaedter, Gina Venolia, Jason Procyk, and Daniel Hawkins. 2016. To Beam or Not to Beam: A Study of Remote Telepresence Attendance at an Academic Conference. In Proceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (San Francisco, California, USA) (CSCW ’16). Association for Computing Machinery, New York, NY, USA, 418–431. https://doi.org/10.1145/2818048.2819922

Digital Library

[34]

Kei Nitta, Keita Higuchi, and Jun Rekimoto. 2014. HoverBall: augmented sports with a flying ball. In Proceedings of the 5th Augmented Human International Conference. 1–4.

Digital Library

[35]

Kei Nitta, Keita Higuchi, Yuichi Tadokoro, and Jun Rekimoto. 2015. Shepherd pass: ability tuning for augmented sports using ball-shaped quadcopter. In Proceedings of the 12th International Conference on Advances in Computer Entertainment Technology. 1–7.

Digital Library

[36]

Nintendo Entertainment Planning & Development Division (EPD) Production Group No.4. 2019. Ring Fit Adventure. Retrieved August 24, 2021 from https://www.nintendo.com/products/detail/ring-fit-adventure-switch/.

[37]

Takuya Nojima, Ngoc Phuong, Takahiro Kai, Toshiki Sato, and Hideki Koike. 2015. Augmented dodgeball: an approach to designing augmented sports. In Proceedings of the 6th Augmented Human International Conference. 137–140.

Digital Library

[38]

Tomoya Ohta, Shumpei Yamakawa, Takashi Ichikawa, and Takuya Nojima. 2014. TAMA: development of trajectory changeable ball for future entertainment. In Proceedings of the 5th Augmented Human International Conference. 1–8.

Digital Library

[39]

Hannah R. M. Pelikan, Amy Cheatle, Malte F. Jung, and Steven J. Jackson. 2018. Operating at a Distance - How a Teleoperated Surgical Robot Reconfigures Teamwork in the Operating Room. Proc. ACM Hum.-Comput. Interact. 2, CSCW, Article 138 (Nov. 2018), 28 pages. https://doi.org/10.1145/3274407

Digital Library

[40]

Irene Rae and Carman Neustaedter. 2017. Robotic Telepresence at Scale. Association for Computing Machinery, New York, NY, USA, 313–324. https://doi.org/10.1145/3025453.3025855

Digital Library

[41]

Alec Rivers, Ilan E. Moyer, and Frédo Durand. 2012. Position-Correcting Tools for 2D Digital Fabrication. ACM Trans. Graph. 31, 4, Article 88 (July 2012), 7 pages. https://doi.org/10.1145/2185520.2185584

Digital Library

[42]

MHD Yamen Saraiji, Tomoya Sasaki, Reo Matsumura, Kouta Minamizawa, and Masahiko Inami. 2018. Fusion: full body surrogacy for collaborative communication. In ACM SIGGRAPH 2018 Emerging Technologies. 1–2.

Digital Library

[43]

Koya Sato, Yuji Sano, Mai Otsuki, Mizuki Oka, and Kazuhiko Kato. 2019. Augmented Recreational Volleyball Court: Supporting the Beginners’ Landing Position Prediction Skill by Providing Peripheral Visual Feedback. In Proceedings of the 10th Augmented Human International Conference 2019. 1–9.

Digital Library

[44]

Roy Shilkrot, Pattie Maes, Joseph A. Paradiso, and Amit Zoran. 2015. Augmented Airbrush for Computer Aided Painting (CAP). ACM Trans. Graph. 34, 2, Article 19 (March 2015), 11 pages. https://doi.org/10.1145/2699649

Digital Library

[45]

Brett Stoll, Samantha Reig, Lucy He, Ian Kaplan, Malte F. Jung, and Susan R. Fussell. 2018. Wait, Can You Move the Robot? Examining Telepresence Robot Use in Collaborative Teams. In Proceedings of the 2018 ACM/IEEE International Conference on Human-Robot Interaction (Chicago, IL, USA) (HRI ’18). Association for Computing Machinery, New York, NY, USA, 14–22. https://doi.org/10.1145/3171221.3171243

Digital Library

[46]

Hang Su, Chenguang Yang, Giancarlo Ferrigno, and Elena De Momi. 2019. Improved human–robot collaborative control of redundant robot for teleoperated minimally invasive surgery. IEEE Robotics and Automation Letters 4, 2 (2019), 1447–1453.

[47]

Ryo Suzuki, Hooman Hedayati, Clement Zheng, James L. Bohn, Daniel Szafir, Ellen Yi-Luen Do, Mark D. Gross, and Daniel Leithinger. 2020. RoomShift: Room-Scale Dynamic Haptics for VR with Furniture-Moving Swarm Robots. Association for Computing Machinery, New York, NY, USA, 1–11. https://doi.org/10.1145/3313831.3376523

Digital Library

[48]

Tim B Swartz. 2009. A new handicapping system for golf. Journal of Quantitative Analysis in Sports 5, 2 (2009).

[49]

Yuchuang Tong and Jinguo Liu. 2021. Review of Research and Development of Supernumerary Robotic Limbs.IEEE CAA J. Autom. Sinica 8, 5 (2021), 929–952.

[50]

Jui-Hung Tu, Chien-Hsun Lee, and Yung-Hsing Chiu. 2005. The analysis of pulling force curves in Tug-of-war. In ISBS-Conference Proceedings Archive.

[51]

Gina Venolia, John Tang, Ruy Cervantes, Sara Bly, George Robertson, Bongshin Lee, and Kori Inkpen. 2010. Embodied Social Proxy: Mediating Interpersonal Connection in Hub-and-Satellite Teams. Association for Computing Machinery, New York, NY, USA, 1049–1058. https://doi.org/10.1145/1753326.1753482

Digital Library

[52]

Lillian Yang, Carman Neustaedter, and Thecla Schiphorst. 2017. Communicating Through A Telepresence Robot: A Study of Long Distance Relationships. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (Denver, Colorado, USA) (CHI EA ’17). Association for Computing Machinery, New York, NY, USA, 3027–3033. https://doi.org/10.1145/3027063.3053240

Digital Library

[53]

Simon Zimmermann, Roi Poranne, and Stelian Coros. 2021. Go Fetch!-Dynamic grasps using Boston Dynamics Spot with external robotic arm. In 2021 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4488–4494.

Digital Library

[54]

Amit Zoran and Joseph A. Paradiso. 2013. FreeD: A Freehand Digital Sculpting Tool. Association for Computing Machinery, New York, NY, USA, 2613–2616. https://doi.org/10.1145/2470654.2481361

Digital Library

Cited By

Uriu DIiyama SOkada THanda TMaekawa AInami M(2023)STAMPER: Human-machine Integrated DrummingExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585619(1-5)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585619
Yamamura NUriu DMuramatsu MKamiyama YKashino ZSakamoto STanaka NTanigawa TOnishi AYoshida SYamanaka SInami M(2023)Social Digital Cyborgs: The Collaborative Design Process of JIZAI ARMSProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581169(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581169

Index Terms

Machine-Mediated Teaming: Mixture of Human and Machine in Physical Gaming Experience
1. Computer systems organization
  1. Embedded and cyber-physical systems
    1. Sensors and actuators
2. Human-centered computing
  1. Human computer interaction (HCI)
    1. Interaction devices
  2. Interaction design

Recommendations

A table tennis game for three players
OZCHI '06: Proceedings of the 18th Australia conference on Computer-Human Interaction: Design: Activities, Artefacts and Environments

Table tennis is a game that can provide healthy exercise and is also a social pastime for players of all ages across the world. However, players have to be collocated to play, and three players cannot usually play at the same time in fair or equitable ...
Human Pacman: a mobile, wide-area entertainment system based on physical, social, and ubiquitous computing

Human Pacman is a novel interactive entertainment system that ventures to embed the natural physical world seamlessly with a fantasy virtual playground by capitalizing on mobile computing, wireless LAN, ubiquitous computing, and motion-tracking ...
Using Machine Learning to Predict Game Outcomes Based on Player-Champion Experience in League of Legends
FDG '21: Proceedings of the 16th International Conference on the Foundations of Digital Games

League of Legends (LoL) is the most widely played multiplayer online battle arena (MOBA) game in the world. An important aspect of LoL is competitive ranked play, which utilizes a skill-based matchmaking system to form fair teams. However, players’ ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems

April 2022

10459 pages

ISBN:9781450391573

DOI:10.1145/3491102

Editors:
Simone Barbosa
PUC-Rio, Brazil
,
Cliff Lampe
University of Michigan, USA
,
Caroline Appert
Université Paris-Saclay, France
,
David A. Shamma
Toyota Research Institute, USA
,
Steven Drucker
Microsoft Research, USA
,
Julie Williamson
University of Glasgow, UK
,
Koji Yatani
University of Tokyo, Japan

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Japan Science and Technology Agency

Conference

CHI '22

Sponsor:

SIGCHI

CHI '22: CHI Conference on Human Factors in Computing Systems

April 29 - May 5, 2022

LA, New Orleans, USA

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
792
Total Downloads

Downloads (Last 12 months)340
Downloads (Last 6 weeks)39

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Uriu DIiyama SOkada THanda TMaekawa AInami M(2023)STAMPER: Human-machine Integrated DrummingExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544549.3585619(1-5)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544549.3585619
Yamamura NUriu DMuramatsu MKamiyama YKashino ZSakamoto STanaka NTanigawa TOnishi AYoshida SYamanaka SInami M(2023)Social Digital Cyborgs: The Collaborative Design Process of JIZAI ARMSProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581169(1-19)Online publication date: 19-Apr-2023
https://dl.acm.org/doi/10.1145/3544548.3581169

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten