Shu Shi
Klara Nahrstedt
ABSTRACT
At the convergence of computer vision, graphics, and multimedia, the emerging 3D video technology promises immersive experiences in a truly seamless environment. However, the requirements of huge network bandwidth and computing resources make it still a big challenge to render 3D video on mobile devices at real-time. In this paper, we present how remote rendering framework can be used to solve the problem. The differences between dynamic 3D video and static graphic models are analyzed. A general proxy-based framework is presented to render 3D video streams on the proxy and transmit the rendered scene to mobile devices over a wireless network. An image-based approach is proposed to enhance 3D interactivity and reduce the interaction delay. Experiments prove that the remote rendering framework can be effectively used for quality 3D video rendering on mobile devices in real time.
- Bumblebee 2 camera. http://www.ptgrey.com/products/bumblebee2/index.asp.Google Scholar
- Chromium. http://chromium.sourceforge.net/.Google Scholar
- Equalizer: Parallel rendering. http://www.equalizergraphics.com/.Google Scholar
- Glintercept. http://glintercept.nutty.org/.Google Scholar
- Gltrace. http://www.hawksoft.com/gltrace/.Google Scholar
- P. Bao and D. Gourlay. Remote walkthrough over mobile networks using 3-D image warping and streaming. Vision, Image and Signal Processing, IEE Proceedings, 151(4):329--336, Aug. 2004.Google Scholar
- A. Boukerche and R. Pazzi. Remote rendering and streaming of progressive panoramas for mobile devices. In Multimedia '06, pages 691--694. ACM, 2006. Google ScholarDigital Library
- C.-F. Chang and S.-H. Ger. Enhancing 3D graphics on mobile devices by image-based rendering. In PCM '02, pages 1105--1111, London, UK, 2002. Google ScholarDigital Library
- H. Kim and et al. An immersive free-viewpoint video system using multiple outer/inner cameras. In 3DPVT, pages 782--789, 2006. Google ScholarDigital Library
- S.-U. Kum and K. Mayer-Patel. Real-time multidepth stream compression. TOMCCAP, 1(2):128--150, 2005. Google ScholarDigital Library
- F. Lamberti and A. Sanna. A streaming-based solution for remote visualization of 3D graphics on mobile devices. IEEE Trans. Vis. Comput. Graph., 13(2):247--260, 2007. Google ScholarDigital Library
- J.-M. Lien and et al. Skeleton-based data compression for multi-camera tele-immersion system. In ISVC'07, pages 714--723, November 2007. Google ScholarDigital Library
- W. R. Mark and et al. Post-rendering 3D warping. In SI3D '97: Proceedings of the 1997 symposium on Interactive 3D graphics, 1997. Google ScholarDigital Library
- W. Matusik and H. Pfister. 3D TV: a scalable system for real-time acquisition, transmission, and autostereoscopic display of dynamic scenes. In SIGGRAPH '04, pages 814--824, 2004. Google ScholarDigital Library
- L. McMillan. An image-based approach to three-dimensional computer graphics. In Ph.D. Dissertation. University of North Carolina at Chapel Hill, Department of Computer Science, 1997. Google ScholarDigital Library
- L. McMillan and G. Bishop. Plenoptic modeling: an image-based rendering system. In SIGGRAPH '95, pages 39--46, 1995. Google ScholarDigital Library
- S. K. Penta and P. J. Narayanan. Compression of multiple depth maps for IBR. The Visual Computer, 21(8-10):611--618, 2005.Google ScholarCross Ref
- A. Redert and et al. ATTEST: Advanced three-dimensional television system technologies. In 3DPVT '02, pages 313--319, June 2002.Google Scholar
- J. Shade and et al. Layered depth images. In SIGGRAPH '98, pages 231--242, 1998. Google ScholarDigital Library
- S. Shi and et al. View-dependent real-time 3D video compression for mobile devices. In Multimedia '08, pages 781--784. ACM, 2008. Google ScholarDigital Library
- H.-Y. Shum, S. B. Kang, and S.-C. Chan. Survey of image-based representations and compression techniques. IEEE Trans. Circuits Syst. Video Techn., 13(11):1020--1037, 2003. Google ScholarDigital Library
- A. Smolic and et al. 3D video and free viewpoint video - technologies, applications and mpeg standards. In ICME, pages 2161--2164. IEEE, 2006.Google ScholarCross Ref
- S. Wurmlin and et al. 3D video recorder: a system for recording and playing free-viewpoint video. Comput. Graph. Forum, 22(2):181--194, 2003.Google ScholarCross Ref
- Z. Yang and et al. TEEVE: The next generation architecture for tele-immersive environment. In ISM '05, pages 112--119, December 2005. Google ScholarDigital Library
- Z. Yang and et al. Real-time 3D video compression for tele-immersive environments. In MMCN '06, 2006.Google Scholar
- Z. Yang and et al. Viewcast: view dissemination and management for multi-party 3D tele-immersive environments. In Multimedia '07. ACM, 2007. Google ScholarDigital Library
Index Terms
- Real-time remote rendering of 3D video for mobile devices
Recommendations
A high-quality low-delay remote rendering system for 3D video
MM '10: Proceedings of the 18th ACM international conference on MultimediaAs an emerging technology, 3D video has shown a great potential to become the next generation media for tele-immersion. However, streaming and rendering this dynamic 3D data in real-time requires tremendous network bandwidth and computing resources. In ...
A real-time remote rendering system for interactive mobile graphics
Special section of best papers of ACM multimedia 2011, and special section on 3D mobile multimediaMobile devices are gradually changing people's computing behaviors. However, due to the limitations of physical size and power consumption, they are not capable of delivering a 3D graphics rendering experience comparable to desktops. Many applications ...
Free-viewpoint video rendering for mobile devices
MIRAGE '13: Proceedings of the 6th International Conference on Computer Vision / Computer Graphics Collaboration Techniques and ApplicationsFree-viewpoint video renderers (FVVR) allow a user to view captured video footage from any position and direction. Despite the obvious appeal of such systems, they have yet to make a major impact on digital entertainment. Current FVVR implementations ...
Comments