skip to main content
10.1145/3581783.3613791acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

IN/ACTive: A Distance-Technology-Mediated Stage for Performer-Audience Telepresence and Environmental Control

Published: 27 October 2023 Publication History

Abstract

The increasing virtualization of the performance process has resulted in passive interactions between performer and audience based on the observing paradigm, but without the presence of in-person staging. We designed a more interactive paradigm of remote performance using a multimedia exhibition strategy where visitors can alter the environment of the performer's location by changing its music and lighting, whereas the performer can create engagement in the exhibition space by affecting a remote robotic arm. We conducted a case study with five participating performers, investigating their expectations, workflows, and perceptions before and after the performance-exhibition, and examining the video footage of their interactions with visitors to understand how they adapt and respond to this remote performance paradigm. We found that the robot arm was perceived as a mediating character with its own distinct identity, that musical changes have implicit behavioral effects on the performance, that lighting manipulations actively changed performer actions during improvisation, and that audiences appear to identify themselves and the robot as integral co-performers in this setup. This work provides insights into how performers learn to engage with audiences in novel distanced spaces and diverse interactive media, generating insights for future virtualized performative interactions beyond the classical stage metaphor.

Supplemental Material

M4V File
IN/ACTIVE is an interactive remote performance setup designed to explore the dynamic relationship between performers and the audience within a new multimedia stage. The presentation video provides an overview of the project, including its background, research questions, and the findings obtained through observations of the exhibitions. Our research contributes to the connections between digital stages and physical spaces, as well as fostering new roles between performers and audience.

References

[1]
Sarah Fdili Alaoui and Jean-Marc Matos. 2021. RCO: Investigating Social and Technological Constraints through Interactive Dance. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1--13. https://doi.org/10.1145/3411764.3445513
[2]
Hesam Alizadeh Talarposhti. 2016. HappyFeet: Embodiments for Joint Remote Dancing. Master's thesis. Graduate Studies. https://doi.org/10.11575/PRISM/26337 Accepted: 2016-09-06T21:41:05Z.
[3]
João Beira, Rodrigo Carvalho, and Sebastian Kox. 2013. Mixed reality immersive design: a study in interactive dance. In Proceedings of the 2013 ACM international workshop on Immersive media experiences (ImmersiveMe '13). Association for Computing Machinery, New York, NY, USA, 45--50. https://doi.org/10.1145/2512142.2512147
[4]
Johannes Birringer. 2004. Interactive dance, the body and the Internet. Journal of Visual Art Practice, Vol. 3, 3 (Jan. 2004), 165--178. https://doi.org/10.1386/jvap.3.3.165/0
[5]
Laura Bissell and Lucy Weir (Eds.). 2021. Performance in a Pandemic. Routledge, London. https://doi.org/10.4324/9781003165644
[6]
John-John Cabibihan, Wing-Chee So, and Soumo Pramanik. 2012. Human-Recognizable Robotic Gestures. IEEE Transactions on Autonomous Mental Development, Vol. 4, 4 (2012), 305--314. https://doi.org/10.1109/TAMD.2012.2208962
[7]
Randy Cohan. 2022. COVID-19's Pandemic's Impact on The Arts: Research Update May 12, 2022. Technical Report. https://www.americansforthearts.org/node/103614
[8]
Ajau Danis, Ain Nadiah Jamaludin, Hayati Adilin Mohd. Abdul Majid, and Khairil Anuar Md Isa. 2016. Body Image Perceptions among Dancers in Urban Environmental Settings. Procedia - Social and Behavioral Sciences, Vol. 222 (June 2016), 855--862. https://doi.org/10.1016/j.sbspro.2016.05.196
[9]
Donna Z Davis and Shelby Stanovsek. 2021. The machine as an extension of the body: When identity, immersion, and interactive design serve as both resource and limitation for the disabled. Human-Machine Communication, Vol. 2 (2021), 121--135.
[10]
Steve Dixon. 2007. Digital Performance: A History of New Media in Theater, Dance, Performance Art, and Installation. https://doi.org/10.7551/mitpress/2429.001.0001
[11]
Hannah J Ellis. 2021. Intersection Between Digital and Dance: The Digital Environment's Impact on the Arts, The. Ph.,D. Dissertation. University of Wyoming. Libraries.
[12]
Hadas Erel, Tzachi Tov, Yoav Kessler, and Oren Zuckerman. 2019. Robots are Always Social: Robotic Movements are Automatically Interpreted as Social Cues. 1--6. https://doi.org/10.1145/3290607.3312758
[13]
Sarah Fdili Alaoui. 2019. Making an Interactive Dance Piece: Tensions in Integrating Technology in Art. In Proceedings of the 2019 on Designing Interactive Systems Conference (DIS '19). Association for Computing Machinery, New York, NY, USA, 1195--1208. https://doi.org/10.1145/3322276.3322289
[14]
Marie Fol. 2021. Virtualised dance ? Digital shifts in artistic practices. https://www.ednetwork.eu/uploads/documents/63/EDN_Virtualized%20Dance%20Full%20Publication.pdf
[15]
Oliver Gingrich, Alain Renaud, and Eugenia Emets. 2013. Kima-a holographic telepresence environment based on cymatic principles. Leonardo, Vol. 46, 4 (2013), 332--343.
[16]
Tom Gorman, Tiina Syrjä, and Mikko Kanninen. 2019. There is a world elsewhere: rehearsing and training through immersive telepresence. Theatre, Dance and Performance Training, Vol. 10, 2 (2019), 208--226.
[17]
Tuomeiciren Heyang and Rose Martin. 2022. Teaching through TikTok: a duoethnographic exploration of pedagogical approaches using TikTok in higher dance education in China and Norway during a global pandemic. Research in Dance Education (Oct. 2022), 1--16. https://doi.org/10.1080/14647893.2022.2114446 Publisher: Routledge.
[18]
Jodi James, Todd Ingalls, Gang Qian, Loren Olsen, Daniel Whiteley, Siew Wong, and Thanassis Rikakis. 2006. Movement-based interactive dance performance. In Proceedings of the 14th ACM international conference on Multimedia. ACM, Santa Barbara CA USA, 470--480. https://doi.org/10.1145/1180639.1180733
[19]
Stephan Jürgens, Nuno N. Correia, and Raul Masu. 2021. The Body Beyond Movement: (Missed) Opportunities to Engage with Contemporary Dance in HCI. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '21). Association for Computing Machinery, New York, NY, USA, 1--9. https://doi.org/10.1145/3430524.3440624
[20]
Gauthier Lafruit, Laurie Van Bogaert, Jaime Sancho Aragon, Michael Panzirsch, Gregoire Hirt, Klaus H. Strobl, and Eduardo Juarez Martinez. 2022. Tele-Robotics VR with Holographic Vision in Immersive Video. In Proceedings of the 1st Workshop on Interactive eXtended Reality (IXR '22). Association for Computing Machinery, New York, NY, USA, 61--68. https://doi.org/10.1145/3552483.3556461
[21]
Susan W. Langdon and Gina Petracca. 2010. Tiny dancer: Body image and dancer identity in female modern dancers. Body Image, Vol. 7, 4 (Sept. 2010), 360--363. https://doi.org/10.1016/j.bodyim.2010.06.005
[22]
RAY LC. 2021. NOW YOU SEE ME, NOW YOU DON'T: revealing personality and narratives from playful interactions with machines being watched. In Proceedings of the Fifteenth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '21). Association for Computing Machinery, New York, NY, USA, 1--7. https://doi.org/10.1145/3430524.3442448
[23]
RAY LC, Aaliyah Alcibar, Alejandro Baez, and Stefanie Torossian. 2020. Machine Gaze: Self-Identification Through Play With a computer Vision-Based Projection and Robotics System. Frontiers in Robotics and AI, Vol. 7 (Dec. 2020), 580835. https://doi.org/10.3389/frobt.2020.580835
[24]
RAY LC, Maurice Benayoun, Permagnus Lindborg, Hongshen Xu, Hin Chung Chan, Ka Man Yip, and Tianyi Zhang. 2021. Power Chess: Robot-to-Robot Nonverbal Emotional Expression Applied to Competitive Play. In 10th International Conference on Digital and Interactive Arts. ACM, Aveiro, Portugal Portugal, 1--11. https://doi.org/10.1145/3483529.3483844
[25]
Jie Li, Xinning Gui, Yubo Kou, and Yukun Li. 2019. Live Streaming as Co-Performance: Dynamics between Center and Periphery in Theatrical Engagement. Proceedings of the ACM on Human-Computer Interaction, Vol. 3 (Nov. 2019), 1--22. https://doi.org/10.1145/3359166
[26]
Vivien Lin, Hui-Chin Yeh, and Nian-Shing Chen. 2022. A Systematic Review on Oral Interactions in Robot-Assisted Language Learning. Electronics, Vol. 11, 2 (Jan. 2022), 290. https://doi.org/10.3390/electronics11020290 Number: 2 Publisher: Multidisciplinary Digital Publishing Institute.
[27]
Andreas H. Marpaung, Sarah M. Brown, and Christine L. Lisetti. 2002. A technical demonstration of Lola, the robot entertainer. In Proceedings of the tenth ACM international conference on Multimedia (MULTIMEDIA '02). Association for Computing Machinery, New York, NY, USA, 91--93. https://doi.org/10.1145/641007.641027
[28]
Eric Mullis. 2013. Dance, Interactive Technology, and the Device Paradigm. Dance Research Journal, Vol. 45, 3 (Dec. 2013), 111--123. https://doi.org/10.1017/S0149767712000290
[29]
Eric B. Nash, Gregory W. Edwards, Jennifer A. Thompson, and Woodrow Barfield. 2000. A Review of Presence and Performance in Virtual Environments. International Journal of Humanâ??Computer Interaction, Vol. 12, 1 (May 2000), 1--41. https://doi.org/10.1207/S15327590IJHC1201_1 Publisher: Taylor & Francis _eprint: https://doi.org/10.1207/S15327590IJHC1201_1.
[30]
Jiazhi Ou, Susan R. Fussell, Xilin Chen, Leslie D. Setlock, and Jie Yang. 2003. Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proceedings of the 5th international conference on Multimodal interfaces (ICMI '03). Association for Computing Machinery, New York, NY, USA, 242--249. https://doi.org/10.1145/958432.958477
[31]
Luigi Pagliarini and Henrik Hautop Lund. 2009. The development of robot art. Artificial Life and Robotics, Vol. 13, 2 (March 2009), 401--405. https://doi.org/10.1007/s10015-008-0620-7
[32]
Hua Peng, Changle Zhou, Huosheng Hu, Fei Chao, and Jing Li. 2015. Robotic Dance in Social Robotics-A Taxonomy. IEEE Transactions on Human-Machine Systems, Vol. 45, 3 (June 2015), 281--293. https://doi.org/10.1109/THMS.2015.2393558 Conference Name: IEEE Transactions on Human-Machine Systems.
[33]
Simon Penny. 2016. Robotics and Art, Computationalism and Embodiment. In Robots and Art: Exploring an Unlikely Symbiosis, Damith Herath, Christian Kroos, and Stelarc (Eds.). Springer, Singapore, 47--65. https://doi.org/10.1007/978-981-10-0321-9_4
[34]
Jon Cedric Roxas, Deborah Richards, Ayse Bilgin, and Nader Hanna. 2018. Exploring the influence of a human-like dancing virtual character on the evocation of human emotion. Behaviour & Information Technology, Vol. 37, 1 (Jan. 2018), 1--15. https://doi.org/10.1080/0144929X.2017.1386714 Publisher: Taylor & Francis.
[35]
Tina M. Schwender, Sarah Spengler, Christina Oedl, and Filip Mess. 2018. Effects of Dance Interventions on Aspects of the Participants' Self: A Systematic Review. Frontiers in Psychology, Vol. 9 (July 2018), 1130. https://doi.org/10.3389/fpsyg.2018.01130
[36]
A. C. Scott. 1975. Reflections on the Aesthetic Background of the Performing Arts of East Asia. Asian Music, Vol. 6, 1/2 (1975), 207--216. https://doi.org/10.2307/833849 Publisher: University of Texas Press.
[37]
Anselm L. Strauss and Juliet M. Corbin. 1998. Basics of qualitative research: techniques and procedures for developing grounded theory 2nd ed ed.). Sage Publications, Thousand Oaks.
[38]
Ryosuke Suzuki and Yoichi Ochiai. 2022. Dance Through Visual Media: The Influence of COVID-19 on Dance Artists. In Universal Access in Human-Computer Interaction. Novel Design Approaches and Technologies, Margherita Antona and Constantine Stephanidis (Eds.). Springer International Publishing, Cham, 360--378.
[39]
Oscar Thörn, Peter Knudsen, and Alessandro Saffiotti. 2020. Human-Robot Artistic Co-Creation: a Study in Improvised Robot Dance. In 2020 29th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN). 845--850. https://doi.org/10.1109/RO-MAN47096.2020.9223446 ISSN: 1944--9437.
[40]
Jutta Toelle and John A Sloboda. 2021. The audience as artist? The audience's experience of participatory music. Musicae Scientiae, Vol. 25, 1 (March 2021), 67--91. https://doi.org/10.1177/1029864919844804 Publisher: SAGE Publications Ltd.
[41]
Lea Vergine. 2000. Body Art and Performance: The Body as Language. Milan: Skira Editore S.p.A.
[42]
Marx Boyuan Wang, Zachary Duer, Scotty Hardwig, Sam Lally, Alayna Ricard, and Myounghoon Jeon. 2022. Echofluid: An Interface for Remote Choreography Learning and Co-creation Using Machine Learning Techniques. In The Adjunct Publication of the 35th Annual ACM Symposium on User Interface Software and Technology. ACM, Bend OR USA, 1--4. https://doi.org/10.1145/3526114.3558708
[43]
Liu Yang. 2022. Influence of Human-Computer Interaction-Based Intelligent Dancing Robot and Psychological Construct on Choreography. Frontiers in Neurorobotics, Vol. 16 (May 2022), 819550. https://doi.org/10.3389/fnbot.2022.819550
[44]
Sun Yuan and Peng Yu. 2018. Can't Help Myself. Retrieved June (2018).
[45]
Wu Zhen and Lian Luan. 2021. Physical World to Virtual Reality - Motion Capture Technology in Dance Creation. Journal of Physics: Conference Series, Vol. 1828, 1 (Feb. 2021), 012097. https://doi.org/10.1088/1742-6596/1828/1/012097
[46]
Qiushi Zhou, Cheng Cheng Chua, Jarrod Knibbe, Jorge Gonçalves, and Eduardo Velloso. 2021. Dance and Choreography in HCI: A Two-Decade Retrospective. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems. ACM, Yokohama Japan, 1--14. https://doi.org/10.1145/3411764.3445804

Cited By

View all
  • (2024)Eternagram: Post-Climate Devastation Text AdventureProceedings of the 17th International Symposium on Visual Information Communication and Interaction10.1145/3678698.3687201(1-2)Online publication date: 11-Dec-2024
  • (2024)THE PRESENT IS IN THE FUTURE: Participatory Generative AI Co-Created Visions as Intangible Cultural HeritageProceedings of the 17th International Symposium on Visual Information Communication and Interaction10.1145/3678698.3687200(1-2)Online publication date: 11-Dec-2024
  • (2024)TIME ENOUGH: Generative AI Visions of Climate Change as Cave Paintings of the FutureProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3672190(608-613)Online publication date: 23-Jun-2024
  • Show More Cited By

Index Terms

  1. IN/ACTive: A Distance-Technology-Mediated Stage for Performer-Audience Telepresence and Environmental Control

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      MM '23: Proceedings of the 31st ACM International Conference on Multimedia
      October 2023
      9913 pages
      ISBN:9798400701085
      DOI:10.1145/3581783
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 27 October 2023

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. environmental control
      2. multimedia performance
      3. performance technology
      4. remote performance
      5. remote robotics

      Qualifiers

      • Research-article

      Conference

      MM '23
      Sponsor:
      MM '23: The 31st ACM International Conference on Multimedia
      October 29 - November 3, 2023
      Ottawa ON, Canada

      Acceptance Rates

      Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)110
      • Downloads (Last 6 weeks)14
      Reflects downloads up to 18 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Eternagram: Post-Climate Devastation Text AdventureProceedings of the 17th International Symposium on Visual Information Communication and Interaction10.1145/3678698.3687201(1-2)Online publication date: 11-Dec-2024
      • (2024)THE PRESENT IS IN THE FUTURE: Participatory Generative AI Co-Created Visions as Intangible Cultural HeritageProceedings of the 17th International Symposium on Visual Information Communication and Interaction10.1145/3678698.3687200(1-2)Online publication date: 11-Dec-2024
      • (2024)TIME ENOUGH: Generative AI Visions of Climate Change as Cave Paintings of the FutureProceedings of the 16th Conference on Creativity & Cognition10.1145/3635636.3672190(608-613)Online publication date: 23-Jun-2024
      • (2024)"Sit on me please": Investigating Perception of Furniture Robotic Movements Using Video PrototypingExtended Abstracts of the CHI Conference on Human Factors in Computing Systems10.1145/3613905.3650778(1-6)Online publication date: 11-May-2024
      • (2024)When Teams Embrace AI: Human Collaboration Strategies in Generative Prompting in a Creative Design TaskProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642133(1-14)Online publication date: 11-May-2024

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media