SAVE: shared augmented virtual environment for real-time mixed reality applications

With head-mounted displays becoming more and more ubiquitous, such headsets are bringing increasingly realistic and immersive experience to the individual users wearing the device. However, other people outside the virtual world are not able to feel or even observe the same vivid virtual environment as the user does. The strong demand of sharing virtual reality experiences among people are hence driving the development of mixed reality (MR) systems that merge real and virtual worlds into the same new environment. In the new world physical and virtual objects exist simultaneously and interact in real time. In this paper we propose Shared Augmented Virtual Environment (SAVE), a mixed reality system that overlays the virtual world with real world objects captured by a Kinect depth camera. We exploit image matting techniques to get the real world data from a green screen background. In the synthetic MR world both real and virtual objects are rendered in real time, where depths from both worlds are handled properly. The advantage of our system is that we connect the virtual and real worlds by a bridge controller mounted on the Kinect. Thereby, we only need to calibrate the whole system once before use. Our results demonstrate that the proposed SAVE system is able to create high quality 1080p live MR footage, enabling the realistic virtual reality experience to be shared among a large number of people at the same time. The proposed method could find potential applications in education, design, entertainment and so on.