ABSTRACT
We explore a future in which people spend considerably more time in virtual reality, even during moments when they transition between locations in the real world. In this paper, we present DreamWalker, a VR system that enables such real-world walking while users explore and stay fully immersed inside large virtual environments in a headset. Provided with a real-world destination, DreamWalker finds a similar path in a pre-authored VR environment and guides the user while real-walking the virtual world. To keep the user from colliding with objects and people in the real-world, DreamWalker's tracking system fuses GPS locations, inside-out tracking, and RGBD frames to 1) continuously and accurately position the user in the real world, 2) sense walkable paths and obstacles in real time, and 3) represent paths through a dynamically changing scene in VR to redirect the user towards the chosen destination. We demonstrate DreamWalker's versatility by enabling users to walk three paths across the large Microsoft campus while enjoying pre-authored VR worlds, supplemented with a variety of obstacle avoidance and redirection techniques. In our evaluation, 8 participants walked across campus along a 15-minute route, experiencing a lively virtual Manhattan that was full of animated cars, people, and other objects.
Supplemental Material
- Thaddeus Beier and Shawn Neely. 1992. Feature-based image metamorphosis. ACM SIGGRAPH Computer Graphics 26, 2 (jul 1992), 35--42. http://dx.doi.org/10.1145/142920.134003Google ScholarDigital Library
- Bigscreen. 2014. https://bigscreenvr.com/. (2014).Google Scholar
- Tomasz Byczkowski and Jochen Lang. 2009. A Stereo-Based System with Inertial Navigation for Outdoor 3D Scanning. In 2009 Canadian Conference on Computer and Robot Vision. IEEE. http://dx.doi.org/10.1109/crv.2009.40Google ScholarDigital Library
- Haiwei Chen, Samantha Chen, and Evan Suma Rosenberg. 2018. Redirected Walking in Irregularly Shaped Physical Environments with Dynamic Obstacles. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE. http://dx.doi.org/10.1109/vr.2018.8446563Google Scholar
- Haiwei Chen and Henry Fuchs. 2017. Supporting free walking in a large virtual environment. In Proceedings of the Computer Graphics International Conference on - CGI textquotesingle17. ACM Press. http://dx.doi.org/10.1145/3095140.3095162Google ScholarDigital Library
- Lung-Pan Cheng, Eyal Ofek, Christian Holz, Hrvoje Benko, and Andrew D. Wilson. 2017. Sparse Haptic Proxy: Touch Feedback in Virtual Environments Using a General Passive Prop. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 3718--3728. http://dx.doi.org/10.1145/3025453.3025753Google Scholar
- Lung-Pan Cheng, Eyal Ofek, Christian Holz, and Andrew D. Wilson. 2019. VRoamer: Generating On-The-Fly VR Experiences While Walking inside Large, Unknown Real-World Building Environments. In 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE.Google Scholar
- Adrian David Cheok. 2010. Human Pacman: A Mobile Augmented Reality Entertainment System Based on Physical, Social, and Ubiquitous Computing. In Art and Technology of Entertainment Computing and Communication. Springer London, 19--57. http://dx.doi.org/10.1007/978--1--84996--137-0_2Google Scholar
- Jan Effertz and Jörn Marten Wille. 2011. Vehicle Architecture and Robotic Perception for Autonomous Driving in Urban Environments. In Experience from the DARPA Urban Challenge. Springer London, 209--239. http://dx.doi.org/10.1007/978-0--85729--772--3_9Google Scholar
- Oculus Go. 2018. Oculus. https://www.oculus.com/go/. (2018).Google Scholar
- Rec Room® - Against Gravity. 2018. https://www.againstgrav.com/rec-room/. (2018).Google Scholar
- Jeremy Hartmann, Christian Holz, Eyal Ofek, and Andrew D. Wilson. 2019. RealityCheck: Blending Virtual Environments with Situated Physical Reality. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (CHI '19). ACM, New York, NY, USA, Article 347, 12 pages. http://dx.doi.org/10.1145/3290605.3300577Google ScholarDigital Library
- Christian Hirt, Markus Zank, and Andreas Kunz. 2017. Real-time wall outline extraction for redirected walking. In Proceedings of the 23rd ACM Symposium on Virtual Reality Software and Technology - VRST textquotesingle17. ACM Press. http://dx.doi.org/10.1145/3139131.3143416Google ScholarDigital Library
- Christian Hirt, Markus Zank, and Andreas Kunz. 2018. Preliminary Environment Mapping for Redirected Walking. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). IEEE. http://dx.doi.org/10.1109/vr.2018.8446262Google Scholar
- M. Keller and F. Exposito. 2018. Game Room Map Integration in Virtual Environments for Free Walking. In 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR). 763--764. http://dx.doi.org/10.1109/VR.2018.8446385Google Scholar
- VIVE? | VIVE Focus Developer Kit. 2018. https://developer.vive.com/eu/vive-focus-for-developer/. (2018).Google Scholar
- Eike Langbehn, Frank Steinicke, Markus Lappe, Gregory F. Welch, and Gerd Bruder. 2018. In the blink of an eye. ACM Transactions on Graphics 37, 4 (jul 2018), 1--11. http://dx.doi.org/10.1145/3197517.3201335Google ScholarDigital Library
- Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431--3440.Google ScholarCross Ref
- Sebastian Marwecki, Maximilian Brehm, Lukas Wagner, Lung-Pan Cheng, Florian textquotesingleFloydtextquotesingle Mueller, and Patrick Baudisch. 2018. VirtualSpace - Overloading Physical Space with Multiple Virtual Reality Users. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems - CHI textquotesingle18. ACM Press. http://dx.doi.org/10.1145/3173574.3173815Google ScholarDigital Library
- Sebastian Marwecki, Andrew D. Wilson, Eyal Ofek, Mar Gonzalez-Franco, and Christian Holz. 2019. Mise-Unseen: Using Eye-Tracking to Hide Virtual Reality Scene Changes in Plain Sight. In Proceedings of the 32nd Annual ACM Symposium on User Interface Software and Technology (UIST '19). ACM, New York, NY, USA, 13.Google ScholarDigital Library
- Quake VR on soccer field. 2019. https://www.youtube.com/watch?v=acEz98Ol8NI. (2019).Google Scholar
- Fallout 4 VR on Steam. 2017. https://store.steampowered.com/agecheck/app/611660/. (2017).Google Scholar
- Brian Peasley and Stan Birchfield. 2013. Replacing Projective Data Association with Lucas-Kanade for KinectFusion. In 2013 IEEE International Conference on Robotics and Automation. IEEE. http://dx.doi.org/10.1109/icra.2013.6630640Google Scholar
- Sharif Razzaque, Zachariah Kohn, and Mary C Whitton. 2001. Redirected walking. 9 (2001), 105--106.Google ScholarDigital Library
- Joseph Redmon. 2013--2016. Darknet: Open Source Neural Networks in C. http://pjreddie.com/darknet/. (2013--2016).Google Scholar
- Occupational Safety, Health Administration, and others. 2016. Improve Tracking of Workplace Injuries and Illnesses. Final rule. Federal register 81, 92 (2016), 29623.Google Scholar
- Anthony Scavarelli and Robert J. Teather. 2017. VR Collide! Comparing Collision-Avoidance Methods Between Co-located Virtual Reality Users. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems - CHI EA textquotesingle17. ACM Press. http://dx.doi.org/10.1145/3027063.3053180Google Scholar
- P. Schmitz, J. Hildebrandt, A. C. Valdez, L. Kobbelt, and M. Ziefle. 2018. You Spin my Head Right Round: Threshold of Limited Immersion for Rotation Gains in Redirected Walking. IEEE Transactions on Visualization and Computer Graphics 24, 4 (April 2018), 1623--1632. http://dx.doi.org/10.1109/TVCG.2018.2793671Google ScholarDigital Library
- Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 1999. Embodied Presence in Virtual Environments. Visual Representations and Interpretations (1999), 269--278. http://dx.doi.org/10.1007/978--1--4471-0563--3_30Google Scholar
- Thomas Schubert, Frank Friedmann, and Holger Regenbrecht. 2001. The Experience of Presence: Factor Analytic Insights. Presence: Teleoperators and Virtual Environments 10, 3 (Jun 2001), 266--281. http://dx.doi.org/10.1162/105474601300343603Google ScholarDigital Library
- Stefania Serafin, Niels C. Nilsson, Erik Sikstrom, Amalia De Goetzen, and Rolf Nordahl. 2013. Estimation of detection thresholds for acoustic based redirected walking techniques. In 2013 IEEE Virtual Reality (VR). IEEE. http://dx.doi.org/10.1109/vr.2013.6549412Google Scholar
- Maeve Serino, Kyla Cordrey, Laura McLaughlin, and Ruth L. Milanaik. 2016. Poké mon Go and augmented virtual reality games. Current Opinion in Pediatrics 28, 5 (oct 2016), 673--677. http://dx.doi.org/10.1097/mop.0000000000000409Google ScholarCross Ref
- Misha Sra, Sergio Garrido-Jurado, and Pattie Maes. 2017. Oasis: Procedurally Generated Social Virtual Spaces from 3D Scanned Real Spaces. IEEE Transactions on Visualization and Computer Graphics (2017), 1--1. http://dx.doi.org/10.1109/tvcg.2017.2762691Google Scholar
- Misha Sra, Sergio Garrido-Jurado, and Chris Schmandt. 2016. Procedurally generated virtual reality from 3D reconstructed physical space. Proceedings of the 22nd ACM Conference on Virtual Reality Software and Technology - VRST '16 (2016). http://dx.doi.org/10.1145/2993369.2993372Google ScholarDigital Library
- Misha Sra, Xuhai Xu, Aske Mottelson, and Pattie Maes. 2018. VMotion. In Proceedings of the 2018 on Designing Interactive Systems Conference 2018 - DIS textquotesingle18. ACM Press. http://dx.doi.org/10.1145/3196709.3196792Google Scholar
- Frank Steinicke, Gerd Bruder, Jason Jerald, Harald Frenz, and Markus Lappe. 2008. Analyses of human sensitivity to redirected walking. In Proceedings of the 2008 ACM symposium on Virtual reality software and technology - VRST textquotesingle08. ACM Press. http://dx.doi.org/10.1145/1450579.1450611Google ScholarDigital Library
- Evan A. Suma, Zachary Lipps, Samantha Finkelstein, David M. Krum, and Mark Bolas. 2012. Impossible Spaces: Maximizing Natural Walking in Virtual Environments with Self-Overlapping Architecture. IEEE Transactions on Visualization and Computer Graphics 18, 4 (April 2012), 555--564. http://dx.doi.org/10.1109/TVCG.2012.47Google ScholarDigital Library
- Qi Sun, Arie Kaufman, Anjul Patney, Li-Yi Wei, Omer Shapira, Jingwan Lu, Paul Asente, Suwen Zhu, Morgan Mcguire, and David Luebke. 2018. Towards virtual reality infinite walking. ACM Transactions on Graphics 37, 4 (jul 2018), 1--13. http://dx.doi.org/10.1145/3197517.3201294Google ScholarDigital Library
- Marvin Teichmann, Michael Weber, Marius Zoellner, Roberto Cipolla, and Raquel Urtasun. 2018. Multinet: Real-time joint semantic reasoning for autonomous driving. In 2018 IEEE Intelligent Vehicles Symposium (IV). IEEE, 1013--1020.Google ScholarDigital Library
- Daniel Thalmann. 2007. Crowd Simulation. American Cancer Society. http://dx.doi.org/10.1002/9780470050118.ecse676Google ScholarDigital Library
- Tom Warren The Verge. 2018. Microsoft is bringing the SharePoint work environment to virtual reality headsets. https://www.theverge.com/2018/5/21/17376422/microsoft- sharepoint-spaces-mixed-reality-virtual- reality-features. (2018).Google Scholar
- Khrystyna Vasylevska, Hannes Kaufmann, Mark Bolas, and Evan A. Suma. 2013. Flexible spaces: Dynamic layout generation for infinite walking in virtual environments. In 2013 IEEE Symposium on 3D User Interfaces (3DUI). IEEE. http://dx.doi.org/10.1109/3dui.2013.6550194Google ScholarCross Ref
- The VOID. 2016. https://www.thevoid.com/. (2016).Google Scholar
- David Waller, Eric Bachmann, Eric Hodgson, and Andrew C. Beall. 2007. The HIVE: A huge immersive virtual environment for research in spatial cognition. Behavior Research Methods 39, 4 (nov 2007), 835--843. http://dx.doi.org/10.3758/bf03192976Google ScholarCross Ref
- Fisher Yu, Vladlen Koltun, and Thomas Funkhouser. 2017. Dilated Residual Networks. In Computer Vision and Pattern Recognition (CVPR).Google Scholar
- Yuhang Zhao, Cynthia L. Bennett, Hrvoje Benko, Edward Cutrell, Christian Holz, Meredith Ringel Morris, and Mike Sinclair. 2018. Enabling People with Visual Impairments to Navigate Virtual Reality with a Haptic and Auditory Cane Simulation. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 116, 14 pages. http://dx.doi.org/10.1145/3173574.3173690Google ScholarDigital Library
- Yaguang Zhu, Baomin Yi, and Tong Guo. 2016. A Simple Outdoor Environment Obstacle Detection Method Based on Information Fusion of Depth and Infrared. Journal of Robotics 2016 (2016), 1--10. http://dx.doi.org/10.1155/2016/2379685Google ScholarDigital Library
Index Terms
- DreamWalker: Substituting Real-World Walking Experiences with a Virtual Reality
Recommendations
Mobile inside-out VR tracking, now available on your phone: extended abstract
SIGGRAPH '18: ACM SIGGRAPH 2018 Appy HourUp to now in mobile Virtual Reality (VR), we have been able to only control the camera orientation with our head. However, premium smartphones already incorporate the essential technology to track user position. Apple ARKit and Google ARCore designed ...
Level-Ups: Motorized Stilts that Simulate Stair Steps in Virtual Reality
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsWe present "Level-Ups", computer-controlled stilts that allow virtual reality users to experience walking up and down steps. Each Level-Up unit is a self-contained device worn like a boot. Its main functional element is a vertical actuation mechanism ...
Making small spaces feel large: infinite walking in virtual reality
SIGGRAPH '15: ACM SIGGRAPH 2015 Emerging TechnologiesOver the past few years, virtual reality has experienced a resurgence. Fueled by a proliferation of consumer-level head-mounted display and motion tracking devices, an unprecedented quantity of immersive experiences and content are available for both ...
Comments