Asreen Rostami
Elena Márquez Segura
ABSTRACT
Designing for interactive performances is challenging both in terms of technology design, and of understanding the interplay between technology, narration, and audience interactions. Bio-sensors and bodily tracking technologies afford new ways for artists to engage with audiences, and for audiences to become part of the artwork. Their deployment raises a number of issues for designers of interactive performances. This paper explores such issues by presenting five design ideas for interactive performance afforded by bio-sensing and bodily tracking (i.e. Microsoft Kinect) developed during two design workshops. We use these ideas, and the related scenarios to discuss three emerging issues namely: temporality of input, autonomy and control, and visibility of input in relation to the deployment of bio-sensors and bodily tracking technologies in the context of interactive performances.
- Marina Abramovic and Klaus Peter Biesenbach. 2010. Marina Abramovi?: The artist is present. The Museum of Modern Art.Google Scholar
- Lars Adde, Jorunn L. Helbostad, Alexander Refsum Jensenius, Gunnar Taraldsen, and Ragnhild Støen. 2009. Using computer-based video analysis in the study of fidgety movements. Early Human Development 85, 9: 541--547.Google ScholarCross Ref
- Sarah Fdili Alaoui, Baptiste Caramiaux, Marcos Serrano, and Frédéric Bevilacqua. 2012. Movement Qualities As Interaction Modality. In Proceedings of the Designing Interactive Systems Conference (DIS '12), 761--769. Google ScholarDigital Library
- Alissa N. Antle, Milena Droumeva, and Greg Corness. 2008. Playing with the Sound Maker: Do Embodied Metaphors Help Children Learn? In Proceedings of the 7th International Conference on Interaction Design and Children (IDC '08), 178--185. Google ScholarDigital Library
- Saskia Bakker, Alissa N. Antle, and Elise Van Den Hoven. 2012. Embodied Metaphors in Tangible Interaction Design. Personal Ubiquitous Comput. 16, 4: 433--449. Google ScholarDigital Library
- Louise Barkhuus and Tobias Jørgensen. 2008. Engaging the Crowd: Studies of Audience-performer Interaction. In CHI '08 Extended Abstracts on Human Factors in Computing Systems (CHI EA '08), 2925--2930. Google ScholarDigital Library
- Louise Barkhuus and Chiara Rossitto. Acting with Technology: Rehearsing for Mixed-Media Live Performances. In Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems (CHI'16). Google ScholarDigital Library
- Steve Benford, Martin Flintham, Adam Drozd, Rob Anastasi, Duncan Rowland, Nick Tandavanitj, Matt Adams, Ju Row-Farr, Amanda Oldroyd, and Jon Sutton. 2004. Uncle Roy All Around You: Implicating the city in a location-based performance. Proc. Advances in Computer Entertainment (ACE 2004) 21: 47.Google Scholar
- Steve Benford, Chris Greenhalgh, Gabriella Giannachi, Brendan Walker, Joe Marshall, and Tom Rodden. 2012. Uncomfortable Interactions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12), 2005--2014. Google ScholarDigital Library
- Kirsten Boehner, Rogério DePaula, Paul Dourish, and Phoebe Sengers. 2007. How emotion is made and measured. International Journal of Human-Computer Studies 65, 4: 275--29 Google ScholarDigital Library
- Francesco Cafaro, Leilah Lyons, Raymond Kang, Josh Radinsky, Jessica Roberts, and Kristen Vogt. 2013. Framed Guessability: Using Embodied Allegories to Increase User Agreement on Gesture Sets. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (TEI '14), 197--204. Google ScholarDigital Library
- Teresa Cerratto-Pargman, Chiara Rossitto, and Louise Barkhuus. 2014. Understanding Audience Participation in an Interactive Theater Performance. In Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, Foundational (NordiCHI '14), 608--617. Google ScholarDigital Library
- Alexis Clay, Gaël Domenger, Julien Conan, Axel Domenger, and Nadine Couture. 2014. Integrating augmented reality to enhance expression, interaction amp; collaboration in live performances: A ballet dance case study. In 2014 IEEE International Symposium on Mixed and Augmented Reality - Media, Art, Social Science, Humanities and Design (ISMAR-MASH'D), 21--29.Google ScholarCross Ref
- Franco Curmi, Maria Angela Ferrario, Jen Southern, and Jon Whittle. 2013. HeartLink: Open Broadcast of Live Biometric Data to Social Networks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 1749--1758. Google ScholarDigital Library
- Scott deLahunta and Frédéric Bevilacqua. 2007. Sharing descriptions of movement. International Journal of Performance Arts and Digital Media 3, 1: 3--16.Google ScholarCross Ref
- J. P. Djajadiningrat, William W. Gaver, and Joep W. Frens. 2000. Interaction Relabelling and Extreme Characters: Methods for Exploring Aesthetic Interactions. In Proceedings of the 3rd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (DIS '00), 66--71. Google ScholarDigital Library
- Nicky Donald and Marco Gillies. 2015. Novel Dramatic and Ludic Tensions Arising from Mixed Reality Performance as Exemplified in Better Than Life. In Interactive Storytelling, Henrik Schoenau-Fog, Luis Emilio Bruni, Sandy Louchart and Sarune Baceviciute (eds.). Springer International Publishing, 297--308. Retrieved November 16, 2016 from http://link.springer.com/chapter/10.1007/978-3-319-27036-4_28Google Scholar
- Paul Dourish. 2004. Where the action is: the foundations of embodied interaction. MIT press.Google Scholar
- Magy Seif El-Nasr. 2007. Interaction, narrative, and drama: Creating an adaptive interactive narrative using performance arts theories. Interaction Studies 8, 2: 209--240.Google ScholarCross Ref
- Moshe Gabel, Ran Gilad-Bachrach, Erin Renshaw, and Assaf Schuster. 2012. Full body gait analysis with Kinect. In 2012 Annual International Conference of the IEEE Engineering in Medicine and Biology Society, 1964--1967.Google ScholarCross Ref
- Kaj Grønbæk, Karen Johanne Kortbek, Claus Møller, Jesper Nielsen, and Liselott Stenfeldt. 2012. Designing Playful Interactive Installations for Urban Environments -- The SwingScape Experience. In Advances in Computer Entertainment, Anton Nijholt, Teresa Romão and Dennis Reidsma (eds.). Springer Berlin Heidelberg, 230--245. Retrieved August 9, 2016 from http://link.springer.com/chapter/10.1007/978-3-642-34292-9_16Google Scholar
- Daniel Hernández, Gabriel Villarrubia, Alberto L. Barriuso, Álvaro Lozano, Jorge Revuelta, and Juan F. De Paz. 2016. Multi Agent Application for Chronic Patients: Monitoring and Detection of Remote Anomalous Situations. In Highlights of Practical Applications of Scalable Multi-Agent Systems. The PAAMS Collection, Javier Bajo, María José Escalona, Sylvain Giroux, Patrycja Hoffa-Dabrowska, Vicente Julián, Paulo Novais, Nayat Sánchez-Pi, Rainer Unland and Ricardo Azambuja-Silveira (eds.). Springer International Publishing, 27--36. Retrieved August 5, 2016 from http://link.springer.com/chapter/10.1007/978-3-319-39387-2_3Google Scholar
- Noura Howell, Laura Devendorf, Rundong (Kevin) Tian, Tomás Vega Galvez, Nan-Wei Gong, Ivan Poupyrev, Eric Paulos, and Kimiko Ryokai. 2016. Biosignals As Social Cues: Ambiguity and Emotional Interpretation in Social Displays of Skin Conductance. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems (DIS '16), 865--870. Google ScholarDigital Library
- Katherine Isbister and Kaho Abe. 2015. Costumes As Game Controllers: An Exploration of Wearables to Suit Social Play. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '15), 691--696. Google ScholarDigital Library
- Ahmad Khawaji, Jianlong Zhou, Fang Chen, and Nadine Marcus. 2015. Using Galvanic Skin Response (GSR) to Measure Trust and Cognitive Load in the Text-Chat Environment. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '15), 1989--1994. Google ScholarDigital Library
- Nicholas D. Lane, Mashfiqui Mohammod, Mu Lin, Xiaochao Yang, Hong Lu, Shahid Ali, Afsaneh Doryab, Ethan Berke, Tanzeem Choudhury, and Andrew T. Campbell. 2011. Bewell: A smartphone application to monitor, model and promote wellbeing. In in Intl. ICST Conf. on Pervasive Computing Technologies for Healthcare.Google Scholar
- Astrid Twenebowa Larssen, Toni Robertson, and Jenny Edwards. 2007. The Feel Dimension of Technology Interaction: Exploring Tangibles Through Movement and Touch. In Proceedings of the 1st International Conference on Tangible and Embedded Interaction (TEI '07), 271--278. Google ScholarDigital Library
- Celine Latulipe, Erin A. Carroll, and Danielle Lottridge. 2011. Love, Hate, Arousal and Engagement: Exploring Audience Responses to Performing Arts. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11), 1845--1854. Google ScholarDigital Library
- Christine La etitia Lisetti and Fatma Nasoz. 2004. Using Noninvasive Wearable Computers to Recognize Human Emotions from Physiological Signals. EURASIP J. Appl. Signal Process. 2004: 1672--1687. Google ScholarDigital Library
- Lian Loke and George Poonkhin Khut. 2011. Surging Verticality: An Experience of Balance. In Proceedings of the Fifth International Conference on Tangible, Embedded, and Embodied Interaction (TEI '11), 237--240. Google ScholarDigital Library
- Lian Loke and Toni Robertson. 2013. Moving and Making Strange: An Embodied Approach to Movement-based Interaction Design. ACM Trans. Comput.-Hum. Interact. 20, 1: 7:1--7:25. Google ScholarDigital Library
- Elena Márquez Segura, Laia Turmo Vidal, and Asreen Rostami. 2016. Bodystorming for movement-based interaction design. Human Technology: An Interdisciplinary Journal on Humans in ICT Environments.Google Scholar
- Elena Márquez Segura, Laia Turmo Vidal, Asreen Rostami, and Annika Waern. Embodied Sketching. In Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems (CHI'16). Google ScholarDigital Library
- Joe Marshall. 2009. I Seek the Nerves Under Your Skin. In Proceedings of the Seventh ACM Conference on Creativity and Cognition (C&C '09), 477--478. Google ScholarDigital Library
- John McCarthy and Peter Wright. 2015. Taking {A}part. MIT Press.Google Scholar
- W. Scott Meador, Timothy J. Rogers, Kevin O'Neal, Eric Kurt, and Carol Cunningham. 2004. Mixing Dance Realities: Collaborative Development of Live-motion Capture in a Performing Arts Environment. Comput. Entertain. 2, 2: 12--1 Google ScholarDigital Library
- Helena M. Mentis and Carolina Johansson. 2013. Seeing Movement Qualities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13), 3375--3384. Google ScholarDigital Library
- Helena M. Mentis, Jarmo Laaksolahti, and Kristina Höök. 2014. My Self and You: Tension in Bodily Sharing of Experience. ACM Trans. Comput.-Hum. Interact. 21, 4: 20:1--20:26. https://doi.org/10.1145/ Google ScholarDigital Library
- Don Norman. 2013. The Design of Everyday Things: Revised and Expanded Edition. Basic Books, New York, New York.Google ScholarDigital Library
- Leif Oppermann, Clemens Putschli, Constantin Brosda, Oleksandr Lobunets, and Fabien Prioville. 2015. The Smartphone Project: An Augmented Dance Performance. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15), 2569--2572. Google ScholarDigital Library
- Alex F. Osborn and Lee Hastings Bristol. 1979. Applied Imagination: Principles and Procedures of Creative Thinking. Charles Scribner's Sons, New York.Google Scholar
- Antti Oulasvirta, Esko Kurvinen, and Tomi Kankainen. 2003. Understanding contexts by being there: case studies in bodystorming. Personal and Ubiquitous Computing 7, 2: 125--134. Google ScholarDigital Library
- Martin Ouwerkerk, Pierre Dandine, Dennis Bolio, Rafal Kocielnik, Johanna Mercurio, Henk Huijgen, and Joyce Westerink. 2013. Wireless Multi Sensor Bracelet with Discreet Feedback. In Proceedings of the 4th Conference on Wireless Health (WH '13), 6:1--6:8. Google ScholarDigital Library
- Galen Panger. 2012. Kinect in the Kitchen: Testing Depth Camera Interactions in Practical Home Environments. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12), 1985--1990. Google ScholarDigital Library
- Tomas Pfister, Xiaobai Li, Guoying Zhao, and Matti Pietikäinen. 2011. Recognising spontaneous facial micro-expressions. In 2011 International Conference on Computer Vision, 1449--1456. Google ScholarDigital Library
- Michalis Raptis, Darko Kirovski, and Hugues Hoppe. 2011. Real-time Classification of Dance Gestures from Skeleton Animation. In Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '11), 147--156. Google ScholarDigital Library
- Omer Rashid, Ian Mullins, Paul Coulton, and Reuben Edwards. 2006. Extending Cyberspace: Location Based Games Using Cellular Phones. Comput. Entertain. 4, 1. Google ScholarDigital Library
- Stuart Reeves. 2011. Designing Interfaces in Public Settings. Springer London, London. Retrieved August 8, 2016 from http://link.springer.com/10.1007/978-0-85729-265-0 Google ScholarDigital Library
- Joseph Butch Rovan, Robert Wechsler, and Frieder Weiss. 2001. Seine hohle Form: Artistic Collaboration in an Interactive Dance and Music Performance Environment. Crossings: eJournal of Art and Technology 1, 2. Retrieved November 16, 2016 from http://crossings.tcd.ie/issues/1.2/Rovan/Google Scholar
- Thecia Schiphorst. 2005. Exhale: Breath Between Bodies. In ACM SIGGRAPH 2005 Electronic Art and Animation Catalog (SIGGRAPH '05), 62--63. Google ScholarDigital Library
- Dennis Schleicher, Peter Jones, and Oksana Kachur. 2010. Bodystorming As Embodied Designing. interactions 17, 6: 47--51. Google ScholarDigital Library
- Vygandas Šimbelis, Anders Lundström, Kristina Höök, Jordi Solsona, and Vincent Lewandowski. 2014. Metaphone: Machine Aesthetics Meets Interaction Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 1--10. Google ScholarDigital Library
- Bart Simon. 2009. Wii are Out of Control: Bodies, Game Screens and the Production of Gestural Excess. Social Science Research Network, Rochester, NY. Retrieved January 17, 2016 from http://papers.ssrn.com/abstract=1354043Google Scholar
- Dag Svanaes. 2013. Interaction Design for and with the Lived Body: Some Implications of Merleau-ponty's Phenomenology. ACM Trans. Comput.-Hum. Interact. 20, 1: 8:1--8:30. Google ScholarDigital Library
- Melanie Swan. 2012. Sensor Mania! The Internet of Things, Wearable Computing, Objective Metrics, and the Quantified Self 2.0. Journal of Sensor and Actuator Networks 1, 3: 217--253.Google ScholarCross Ref
- Emmanuel Munguia Tapia, Stephen S. Intille, William Haskell, Kent Larson, Julie Wright, Abby King, and Robert Friedman. 2007. Real-Time Recognition of Physical Activities and Their Intensities Using Wireless Accelerometers and a Heart Rate Monitor. In 2007 11th IEEE International Symposium on Wearable Computers, 37--40. Google ScholarDigital Library
- Paul Tennent, Stuart Reeves, Steve Benford, Brendan Walker, Joe Marshall, Patrick Brundell, Rupert Meese, and Paul Harter. 2012. The Machine in the Ghost: Augmenting Broadcasting with Biodata. In CHI '12 Extended Abstracts on Human Factors in Computing Systems (CHI EA '12), 91--100. Google ScholarDigital Library
- Jakob Tholander and Stina Nylander. 2015. Snot, Sweat, Pain, Mud, and Snow: Performance and Experience in the Use of Sports Watches. 2913--2922. Google ScholarDigital Library
- Bin Umino, Jeffrey Scott Longstaff, and Asako Soga. 2009. Feasibility study for ballet e-learning: automatic composition system for ballet enchaînement with online 3D motion data archive. Research in Dance Education 10, 1: 17--32.Google ScholarCross Ref
- Jordan Beth Vincent, Kim Vincs, and J. McCormick. 2015. Splitting centre?: directing attention in trans-media dance performance. ISEA 2015?: Proceedings of the 21st International Symposium on Electronic Art: 1--1.Google Scholar
- Chen Wang, Erik N. Geelhoed, Phil P. Stenton, and Pablo Cesar. 2014. Sensing a Live Audience. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14), 1909--1912. Google ScholarDigital Library
- Qiao Wang, Pavan Turaga, Grisha Coleman, and Todd Ingalls. 2014. SomaTech: An Exploratory Interface for Altering Movement Habits. In Proceedings of the Extended Abstracts of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI EA '14), 1765--1770. Google ScholarDigital Library
- Lisa N. Woodside, V. K. Kumar, and Ronald J. Pekala. 1997. Monotonous Percussion Drumming and Trance Postures: A Controlled Evaluation of Phenomenological Effects. Anthropology of Consciousness 8, 2--3: 69--87.Google ScholarCross Ref
- Kuan-Ju Wu, Mark D. Gross, and Mark Baskinger. 2012. Giffi: A Gift for Future Inventors. In Proceedings of the Sixth International Conference on Tangible, Embedded and Embodied Interaction (TEI '12), 335--336. Google ScholarDigital Library
- Yan Zhang, Lei Wang, Jing Zhu, Yadong Hu, Wanli Xing, and Jing Cheng. 2011. Real-time monitoring of extracellular matrix-mediated PC12 cell attachment and proliferation using an electronic biosensing device. Biotechnology Letters 34, 2: 397--404.Google ScholarCross Ref
- Touching a Stranger: Designing for Engaging Experience in Embodied Interaction. International Journal of Dsign. Retrieved January 17, 2016 from http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/976Google Scholar
- Thinking Through the Body | Body-focussed Interactive Art and Somatics. Retrieved January 17, 2016 from http://thinkingthroughthebody.net/Google Scholar
Index Terms
- Bio-Sensed and Embodied Participation in Interactive Performance
Recommendations
Understanding audience participation in an interactive theater performance
NordiCHI '14: Proceedings of the 8th Nordic Conference on Human-Computer Interaction: Fun, Fast, FoundationalThis article presents an empirical study investigating audience participation in an interactive theater performance. During the performance, audience members were enticed to act upon and contribute to the performance by sharing their opinions, emotions, ...
Performing with Technology
TEI '17: Proceedings of the Eleventh International Conference on Tangible, Embedded, and Embodied InteractionThis research explores the use of technology in Mixed-Reality Performance, and how novel interactive technologies (VR, IoT, Bio-Data) can be used in designing such performances. Additionally, I explore the potential of these technologies to increase ...
Workshop Transmedia Journalism and Interactive Documentary in Dialogue
Interactive StorytellingAbstractThis half-day workshop promotes a hands-on approach to recent developments in transmedia storytelling and its application to journalism and interactive documentary (iDoc). Transmedia journalism, as well as any other application of transmedia ...
Comments