Michal Joachimczak
ABSTRACT
We present a demo of low-cost mixed reality telepresence system that performs real-time 3D reconstruction of a person or an object and wirelessly transmits the reconstructions to Microsoft's HoloLens head mounted display at frame rates perceived as smooth. A reconstructed frame is represented as a polygonal mesh with polygons textured with high definition data obtained from RGB cameras. Each frame is compressed and sent to HoloLens, so that it can be locally rendered by its GPU, minimizing latency of reacting to head movement. Owing to HoloLens half-translucent displays, the system creates an appearance of remote object being part of user's physical environment. The system can run on a relatively low-cost commodity hardware, such as Kinect sensors and in the most basic scenario can produce smooth frame rates of a single multicore laptop.
- Sergio O. Escolano, Christoph Rhemann, Sean Fanello, Wayne Chang, Adarsh Kowdle, et al. 2016. Holoportation: Virtual 3D Teleportation in Real-time. In Proceedings of the 29th Annual Symposium on User Interface Software and Technology (UIST ’16). ACM, New York, NY, USA, 741–754. Google Scholar
Digital Library
- 2984517Google Scholar
- Yunhong Gu and Robert L. Grossman. 2007. UDT: UDP-based data transfer for high-speed wide area networks. Computer Networks 51, 7 (16 May 2007), 1777–1799. Google ScholarDigital Library
- Andrew Maimone, Xubo Yang, Nate Dierk, Andrei State, Mingsong Dou, and Henry Fuchs. 2013. General-purpose telepresence with head-worn optical seethrough displays and projector-based lighting. In 2013 IEEE Virtual Reality (VR). IEEE, 23–26.Google Scholar
- Radu B. Rusu and Steve Cousins. 2011. 3D is here: Point Cloud Library (PCL). In 2011 IEEE International Conference on Robotics and Automation. IEEE, 1–4.Google Scholar
- Jessica Tyrrell, Samuel E. Jones, Robin Beaumont, Christina M. Astley, Rebecca Lovell, et al. 2016. Height, body mass index, and socioeconomic status: mendelian randomisation study in UK Biobank. BMJ 352 (08 March 2016), i582+. Abstract 1 Introduction 2 System architecture 3 Demonstration ReferencesGoogle Scholar
Index Terms
- Real-time mixed-reality telepresence via 3D reconstruction with HoloLens and commodity depth sensors
Recommendations
Real-time 3D video avatar in mixed reality: an implementation for immersive telecommunication
Symposium: virtual reality simulationThis article presents an implementation of a real-time dynamic 3D avatar from multiview cameras for immersive telecommunication. Immersive telecommunication is a new challenging field that enables a user to share a virtual space with remote participants,...
A mixed reality telepresence system with limited DOF motion base and immersive display
ACE '09: Proceedings of the International Conference on Advances in Computer Entertainment TechnologyThis paper describes a mixed reality (MR) telepresence system for a ride to provide users with a highly realistic sensation. To make a realistic scene in a virtual environment, it is necessary to combine visual information with a reproduction of the ...
Real time 3D avatar for interactive mixed reality
VRCAI '04: Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industryThis paper presents real-time reconstruction of dynamic 3D avatar for interactive mixed reality. In computer graphics, one of the main goals is the combination of virtual scenes with real-world scenes. However, the views of the real world objects are ...
Comments