Michael Nebeling
ABSTRACT
Immersive authoring is an increasingly popular technique to design AR/VR scenes because design and testing can be done concurrently. Most existing systems, however, are single-user and limited to either AR or VR, thus constrained in the interaction techniques. We present XRDirector, a role-based collaborative immersive authoring system that enables designers to freely express interactions using AR and VR devices as puppets to manipulate virtual objects in 3D physical space. In XRDirector, we adapt roles known from filmmaking to structure the authoring process and help coordinate multiple designers in immersive authoring tasks. We study how novice AR/VR creators can take advantage of the roles and modes in XRDirector to prototype complex scenes with animated 3D characters, light effects, and camera movements, and also simulate interactive system behavior in a Wizard of Oz style. XRDirector's design was informed by case studies around complex 3D movie scenes and AR/VR games, as well as workshops with novice AR/VR creators. We show that XRDirector makes it easier and faster to create AR/VR scenes without the need for coding, characterize the issues in coordinating designers between AR and VR, and identify the strengths and weaknesses of each role and mode to mitigate the issues.
Supplemental Material
- Raphael Anderegg, Loïc Ciccone, and Robert W.Sumner. 2018. PuppetPhone: puppeteering virtualcharacters using a smartphone. InProc. MIG. 5:1--5:6.Google Scholar
- Connelly Barnes, David E. Jacobs, Jason Sanders, Dan B. Goldman, Szymon Rusinkiewicz, Adam Finkelstein, and Maneesh Agrawala. 2008. Video puppetry: a performative interface for cutout animation. TOG 27, 5 (2008), 124:1--124:9.Google ScholarDigital Library
- Mark Billinghurst and Hirokazu Kato. 1999. Collaborative mixed reality. In Proc. ISMAR. 261--284.Google ScholarCross Ref
- Lung-Pan Cheng, Patrick Lühne, Pedro Lopes, Christoph Sterz, and Patrick Baudisch. 2014. Haptic turk: a motion platform based on people. In Proc. CHI. DOI: http://dx.doi.org/10.1145/2556288.2557101Google ScholarDigital Library
- Lung-Pan Cheng, Sebastian Marwecki, and Patrick Baudisch. 2017. Mutual Human Actuation. In Proc. UIST. 797--805.Google ScholarDigital Library
- Lung-Pan Cheng, Thijs Roumen, Hannes Rantzsch, Sven Köhler, Patrick Schmidt, Robert Kovacs, Johannes Jasper, Jonas Kemper, and Patrick Baudisch. 2015. TurkDeck: Physical Virtual Reality Based on People. In Proc. UIST. 417--426.Google ScholarDigital Library
- Richard C. Davis, Brien Colwell, and James A. Landay. 2008. K-sketch: a 'kinetic' sketch pad for novice animators. In Proc. CHI. 413--422.Google Scholar
- Mira Dontcheva, Gary D. Yngve, and Zoran Popovic. 2003. Layered acting for character animation. TOG 22, 3 (2003), 409--416. DOI: http://dx.doi.org/10.1145/882262.882285Google ScholarDigital Library
- Steven Dow, Jaemin Lee, Christopher Oezbek, Blair MacIntyre, Jay David Bolter, and Maribeth Gandy. 2005a. Wizard of Oz interfaces for mixed reality applications. In Proc. CHI Extended Abstracts. 1339--1342.Google ScholarDigital Library
- Steven Dow, Blair MacIntyre, Jaemin Lee, Christopher Oezbek, Jay David Bolter, and Maribeth Gandy. 2005b. Wizard of Oz support throughout an iterative design process. IEEE Pervasive Computing 4, 4 (2005), 18--26.Google ScholarDigital Library
- Barrett Ens, Joel Lanir, Anthony Tang, Scott Bateman, Gun Lee, Thammathip Piumsomboon, and Mark Billinghurst. 2019. Revisiting collaboration through mixed reality: The evolution of groupware. IJHCS 131 (2019).Google Scholar
- Jamie Feltham. 2017. Disney Is Using VR To Help Film The Lion King Remake. Available from uploadVR at https://uploadvr.com/disney-using-vr-film-lion-king-remake/. (July 2017).Google Scholar
- Maribeth Gandy and Blair MacIntyre. 2014. Designer's augmented reality toolkit, ten years later: implications for new media authoring tools. In Proc. UIST.Google ScholarDigital Library
- Taejin Ha, Woontack Woo, Youngho Lee, Junhun Lee, Jeha Ryu, Hankyun Choi, and Kwanheng Lee. 2010. ARtalet: tangible user interface based immersive augmented reality authoring tool for Digilog book. In Proc. ISUVR. 40--43.Google ScholarDigital Library
- Robert Held, Ankit Gupta, Brian Curless, and Maneesh Agrawala. 2012. 3D puppetry: a kinect-based interface for 3D animation.. In Proc. UIST. 423--434.Google ScholarDigital Library
- Andrew J. Hunsucker, Kelly McClinton, Jennifer Wang, and Erik Stolterman. 2017. Augmented Reality Prototyping For Interaction Design Students. In Proc. CHI Extended Abstracts. 1018--1023.Google ScholarDigital Library
- Annie Kelly, R. Benjamin Shapiro, Jonathan de Halleux, and Thomas Ball. 2018. ARcadia: A Rapid Prototyping Platform for Real-time Tangible Interfaces. In Proc. CHI.Google Scholar
- Boriana Koleva, Ian Taylor, Steve Benford, Mike Fraser, Chris Greenhalgh, Holger Schnädelbach, Dirk vom Lehn, Christian Heath, Ju Row-Farr, and Matt Adams. 2001. Orchestrating a mixed reality performance. In Proc. CHI. 38--45.Google ScholarDigital Library
- Walter S. Lasecki, Juho Kim, Nick Rafter, Onkur Sen, Jeffrey P. Bigham, and Michael S. Bernstein. 2015. Apparition: Crowdsourced User Interfaces that Come to Life as You Sketch Them. In Proc. CHI.Google Scholar
- Gun A. Lee, Claudia Nelles, Mark Billinghurst, and Gerard Jounghyun Kim. 2004. Immersive Authoring of Tangible Augmented Reality Applications. In Proc. ISMAR. 172--181.Google ScholarDigital Library
- Sang Won Lee, Yujin Zhang, Isabelle Wong, Yi Wei Yang, Stephanie D. O'Keefe, and Walter S. Lasecki. 2017. SketchExpress: Remixing Animations for More Effective Crowd-Powered Prototyping of Interactive Interfaces. In Proc. UIST. 817--828.Google Scholar
- Germán Leiva and Michel Beaudouin-Lafon. 2018. Montage: A Video Prototyping System to Reduce Re-Shooting and Increase Re-Usability. In Proc. UIST. 675--682.Google ScholarDigital Library
- David Lindlbauer and Andy D. Wilson. 2018. Remixed Reality: Manipulating Space and Time in Augmented Reality. In Proc. CHI. 129.Google Scholar
- Blair MacIntyre, Maribeth Gandy, Steven Dow, and Jay David Bolter. 2004. DART: a toolkit for rapid design exploration of augmented reality experiences. In Proc. UIST.Google ScholarDigital Library
- Jens Müller, Roman Rädle, and Harald Reiterer. 2016. Virtual Objects as Spatial Cues in Collaborative Mixed Reality Environments: How They Shape Communication Behavior and User Task Load. In Proc. CHI. 1245--1249.Google ScholarDigital Library
- Michael Nebeling and Katy Madier. 2019. 360proto: Making Interactive Virtual Reality Augmented Reality Prototypes from Paper. In Proc. CHI.Google ScholarDigital Library
- Michael Nebeling, Janet Nebeling, Ao Yu, and Rob Rumble. 2018. ProtoAR: Rapid Physical-Digital Prototyping of Mobile Augmented Reality Applications. In Proc. CHI.Google ScholarDigital Library
- Michael Nebeling and Maximilian Speicher. 2018. The Trouble with Augmented Reality/Virtuality Reality Authoring Tools. In Proc. ISMAR Adjunct.Google ScholarCross Ref
- Michael Nitsche and Sanjeev Nayak. 2012. Cell Phone Puppets: Turning Mobile Phones into Performing Objects. In Proc. ICEC. 363--372.Google ScholarDigital Library
- Ivan Poupyrev, Mark Billinghurst, Suzanne Weghorst, and Tadao Ichikawa. 1996. The Go-Go Interaction Technique: Non-Linear Mapping for Direct Manipulation in VR. In Proc. UIST. 79--80.Google ScholarDigital Library
- Dieter Schmalstieg, Anton Fuhrmann, Gerd Hesina, Zsolt Szalavári, L Miguel Encarnaçao, Michael Gervautz, and Werner Purgathofer. 2002. The studierstube augmented reality project. Presence: Teleoperators & Virtual Environments 11, 1 (2002), 33--54.Google ScholarDigital Library
- Harrison Jesse Smith and Michael Neff. 2018. Communication Behavior in Embodied Virtual Reality. In Proc. CHI. 289.Google ScholarDigital Library
- Maximilian Speicher, Brian D. Hall, and Michael Nebeling. 2019. What is Mixed Reality?. In Proc. CHI.Google ScholarDigital Library
- Maximilian Speicher and Michael Nebeling. 2018. GestureWiz: A Human-Powered Gesture Design Environment for User Interface Prototypes. In Proc. CHI.Google ScholarDigital Library
- Jia Wang, Owen Leach, and Robert W. Lindeman. 2013. DIY World Builder: An immersive level-editing system. In Proc. 3DUI. 195--196.Google Scholar
Index Terms
- XRDirector: A Role-Based Collaborative Immersive Authoring System
Recommendations
360proto: Making Interactive Virtual Reality & Augmented Reality Prototypes from Paper
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing SystemsWe explore 360 paper prototyping to rapidly create AR/VR prototypes from paper and bring them to life on AR/VR devices. Our approach is based on a set of emerging paper prototyping templates specifically for AR/VR. These templates resemble the key ...
FlowMatic: An Immersive Authoring Tool for Creating Interactive Scenes in Virtual Reality
UIST '20: Proceedings of the 33rd Annual ACM Symposium on User Interface Software and TechnologyImmersive authoring is a paradigm that makes Virtual Reality (VR) application development easier by allowing programmers to create VR content while immersed in the virtual environment. In this paradigm, programmers manipulate programming primitives ...
Immersive authoring of Tangible Augmented Reality content: A user study
Immersive authoring refers to the style of programming or developing content from within the targetexecutable environment. Immersive authoring is important for fields such as augmented reality (AR) in which interaction usability and user perception of ...
Comments