skip to main content
10.1145/1878083.1878087acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

A real-time performance system for virtual theater

Published: 29 October 2010 Publication History

Abstract

The idea of combining virtual reality technology and theatrical tradition to create virtual plays has captured artists' imaginations for some time. Using conventional technology, the use of virtual characters in a theatrical performance often integrates the predefined animations of virtual actors into the theater scene, resulting in a performance that can feel stilted and unresponsive due to its pre-programmed nature. This paper proposes a new system that allows actors to animate virtual characters in real time, resulting in a more flexible and interactive theatrical performance experience. Actors are sequestered at a remote site, invisible to the audience, and are digitized by a motion capture system. Using camera feeds to provide the remote actors with information about the behavior of the live actors and audience in the theater, the remote actors can adapt their virtual counterparts' behavior to react to live events in real-time, giving the illusion to the audience that the virtual characters are responsive to their actions.
The system integrates display peripherals, networked cameras, real-time motion capture system, gesture recognition, and a virtual environment development suite (Virtools) to create a true virtual theatrical performance environment. In the paper, we will present the various concepts developed so far and an example of a virtual theatrical performance called Trickster at the Intersection that was presented during Smart Graphics 2010 at Canada's Banff Centre.

References

[1]
S. C. Ahn, I.-J. Kim, H.-G. Kim, Y.-M. Kwon, and H. Ko. Audience interaction for virtual reality theater and its implementation. In VRST '01: Proceedings of the ACM symposium on Virtual reality software and technology, pages 41--45, New York, NY, USA, 2001. ACM.
[2]
J. Chai and J. K. Hodgins. Performance animation from low-dimensional control signals. ACM Trans. Graph., 24(3):686--696, 2005.
[3]
A. D. Cheok, W. Weihua, X. Yang, S. Prince, F. S. Wan, M. Billinghurst, and H. Kato. Interactive theater experience in embodied + wearable mixed reality space. In ISMAR '02: Proceedings of the 1st International Symposium on Mixed and Augmented Reality, page 59, Washington, DC, USA, 2002. IEEE Computer Society.
[4]
S. P. Dow, M. Mehta, B. MacIntyre, and M. Mateas. Eliza meets the wizard-of-oz: evaluating social acceptability. In CHI '10: Proceedings of the 28th international conference on Human factors in computing systems, pages 547--556, New York, NY, USA, 2010. ACM.
[5]
L. Engler and C. Fijan. Making puppets come alive: how to learn and teach hand puppetry. Dover Publications, 1997.
[6]
G. Giesekam. staging the screen: the use of film an video in theatre. Palgrave Macmillan, 2008.
[7]
B. Hayes-Roth and L. Brownston. Multiagent collaboration in directed improvisation. In Proceedings of the First International Conference on Multi-Agent Systems, pages 148--154, 1995.
[8]
Z. Hendricks, G. Marsden, and E. Blake. A meta-authoring tool for specifying interactions in virtual reality environments. In AFRIGRAPH '03: Proceedings of the 2nd international conference on Computer graphics, virtual Reality, visualisation and interaction in Africa, pages 171--180, New York, NY, USA, 2003. ACM.
[9]
M. Mateas and A. Stern. Facade: An experiment in building a fully-realized interactive drama. In Game developer's Conference: Game Design Track, 2003.
[10]
M. v. d. P. Matthew Thorne, David Burke. Motion doodle: An interface for sketching character motion. In Proc. of SIGGRAPH '04 (Special issue of ACM Transactions on Graphics), 2004.
[11]
W. S. Meador, T. J. Rogers, K. O'Neal, E. Kurt, and C. Cunningham. Mixing dance realities: collaborative development of live-motion capture in a performing arts environment. Comput. Entertain., 2(2):12--12, 2004.
[12]
C. P. N. Badler. Simulating Humans: Computer Graphics, Animation and Control. Oxford University Press, 1993.
[13]
D. Ninomiya, K. Miyazaki, and R. Nakatsu. Networked virtual marionette theater. In KES '08: Proceedings of the 12th international conference on Knowledge-Based Intelligent Information and Engineering Systems, Part I, pages 440--447, Berlin, Heidelberg, 2008. Springer-Verlag.
[14]
K. Perlin and A. Goldberg. Improv: a system for scripting interactive actors in virtual worlds. In SIGGRAPH '96: Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, pages 205--216, New York, NY, USA, 1996. ACM.
[15]
C. Reeve. Presence in virtual theater. Presence: Teleoper. Virtual Environ., 9(2):209--213, 2000.
[16]
M. Shiba, A. Soga, and J. Salz. A virtual performance system and its application on a noh stage. In VRST '09:Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, pages 267--268, New York, NY, USA, 2009. ACM.
[17]
T. Shiratori and J. K. Hodgins. Accelerometer-based user interfaces for the control of a physically simulated character. ACM Transactions on Graphics (SIGGRAPH Asia 2008), 27(5), 2008.
[18]
S. Strassman. Desktop Theater: Automatic generation of Expressive Animation, PhD thesis. MIT Media Lab, 1991.
[19]
S. Ullrich, H. Prendinger, and M. Ishizuka. Mpml3d: agent authoring language for virtual worlds. In ACE '08: Proceedings of the 2008 International Conference on Advances in Computer Entertainment Technology, pages 134--137, New York, NY, USA, 2008. ACM.
[20]
K. Yin and D. K. Pai. Footsee: an interactive animation system. In SCA '03: Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, pages 329--338, Aire-la-Ville, Switzerland, Switzerland, 2003. Eurographics Association.
[21]
P. Zhao and M. van de Panne. User interfaces for interactive control of physics-based 3d characters. In I3D '05: Proceedings of the 2005 symposium on Interactive 3D graphics and games, pages 87--94, New York, NY, USA, 2005. ACM.

Cited By

View all
  • (2024)Enhancing Script Work and Memorization for Amateur Actors Using Virtual RealityAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685421(1-5)Online publication date: 13-Oct-2024
  • (2024)Process, Roles, Tools, and Team: Understanding the Emerging Medium of Virtual Reality TheatreProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642741(1-14)Online publication date: 11-May-2024
  • (2022)Research on Virtual Interactive Animation Design System Based on Deep LearningComputational Intelligence and Neuroscience10.1155/2022/50353692022Online publication date: 1-Jan-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SMVC '10: Proceedings of the 2010 ACM workshop on Surreal media and virtual cloning
October 2010
76 pages
ISBN:9781450301756
DOI:10.1145/1878083
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 October 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. audience interaction
  2. gesture recognition
  3. motion capture
  4. real-time 3d graphics
  5. virtual theater

Qualifiers

  • Research-article

Conference

MM '10
Sponsor:
MM '10: ACM Multimedia Conference
October 29, 2010
Firenze, Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)17
  • Downloads (Last 6 weeks)2
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Enhancing Script Work and Memorization for Amateur Actors Using Virtual RealityAdjunct Proceedings of the 2024 Nordic Conference on Human-Computer Interaction10.1145/3677045.3685421(1-5)Online publication date: 13-Oct-2024
  • (2024)Process, Roles, Tools, and Team: Understanding the Emerging Medium of Virtual Reality TheatreProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642741(1-14)Online publication date: 11-May-2024
  • (2022)Research on Virtual Interactive Animation Design System Based on Deep LearningComputational Intelligence and Neuroscience10.1155/2022/50353692022Online publication date: 1-Jan-2022
  • (2022)Stage Together: Remote Rehearsal of Theater BlockingProceedings of the 8th International Conference on Movement and Computing10.1145/3537972.3537977(1-5)Online publication date: 22-Jun-2022
  • (2022)Effects of Augmenting Real-Time Biofeedback in An Immersive VR Performance2022 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)10.1109/ISMAR-Adjunct57072.2022.00159(751-756)Online publication date: Oct-2022
  • (2021)Keeping the living bond between actors and remote audiences in distributed virtual theaterProceedings of the 10th International Conference on Digital and Interactive Arts10.1145/3483529.3483683(1-12)Online publication date: 13-Oct-2021
  • (2021)Performing with Me: Enhancing Audience-Performer Interaction in An Immersive Virtual PlayExtended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411763.3451686(1-6)Online publication date: 8-May-2021
  • (2020)How a Hyper-actor directs Avatars in Virtual Shadow TheaterProceedings of the 7th International Conference on Movement and Computing10.1145/3401956.3404234(1-9)Online publication date: 15-Jul-2020
  • (2020)Investigating Roleplaying and Identity Transformation in a Virtual Reality Narrative ExperienceProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376762(1-13)Online publication date: 21-Apr-2020
  • (2019)VR as a Content Creation Tool for Movie Previsualisation2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR)10.1109/VR.2019.8798181(303-311)Online publication date: Mar-2019
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media