Immersive interactive reality: Internet-based on-demand VR for cultural presentation

Abstract

This paper presents an Internet-based virtual reality technology, called panoramic broadcasting (PanoCAST) where multiple viewers share an experience yet each having full control of what they see independent from other viewers. Our solution was developed for telepresence-based cultural presentation and entertainment services. The core architecture involves a compact spherical vision system that compresses and transmits data from multiple digital video sources to a central host computer, which in turn distributes the recorded information among multiple render- and streaming servers for personalized viewing over the Internet or mobile devices. In addition, using advanced computer vision, tracking and animation features, the PanoCAST architecture introduces the notion of Clickable Content Management (CCM), where each visual element in the image becomes a source for providing further information, educational content and cultural detail. Key contributions of our application to advance the state-of-the-art include bringing streaming panoramic video onto mobile platforms, an advanced tracking interface to turn visual elements into sources of interaction, physical simulation to combine the benefits of panoramic video with that of 3D models and animated, photo-realistic faces to help users express their emotions in shared online virtual cultural experiences as well as a feedback mechanism in such environments. Therefore, we argue that the PanoCAST system offers a low-cost and economical solution for personalized content distribution and as such it can serve as a unified basis for novel applications many of which are demonstrated in this paper.