skip to main content
research-article

A novel 3D video transcoding scheme for adaptive 3D video transmission to heterogeneous terminals

Published: 16 October 2012 Publication History

Abstract

Three-dimensional video (3DV) is attracting many interests with its enhanced viewing experience and more user driven features. 3DV has several unique characteristics different from 2D video: (1) It has a much larger amount of data captured and compressed, and corresponding video compression techniques can be much more complicated in order to explore data redundancy. This will lead to more constraints on users' network access and computational capability, (2) Most users only need part of the 3DV data at any given time, while the users' requirements exhibit large diversity, (3) Only a limited number of views are captured and transmitted for 3DV. View rendering is thus necessary to generate virtual views based on the received 3DV data. However, many terminal devices do not have the functionality to generate virtual views. To enable 3DV experience for the majority of users with limited capabilities, adaptive 3DV transmission is necessary to extract/generate the required data content and represent it with supported formats and bitrates for heterogeneous terminal devices. 3DV transcoding is an emerging and effective technique to achieve desired adaptive 3DV transmission. In this article, we propose the first efficient 3DV transcoding scheme that can obtain any desired view, either an encoded one or a virtual one, and compress it with more universal H.264/AVC. The key idea of the proposed scheme is to appropriately utilize motion information contained in the bitstream to generate candidate motion information. Original information of both the desired view and reference views are used to obtain this candidate information and a proper motion refinement process is carried out for certain blocks. Simulation results show that, compared to the straightforward cascade algorithm, the proposed scheme is able to output compressed bitstream of the required view with significantly reduced complexity while incurring negligible performance loss. Such a 3DV transcoding can be applied to most gateways that usually have constraints on computational complexity and time delay.

References

[1]
Ahmand, I., Wei, X., Sun, Y., and Zhang, Y.-Q. 2005. Video transcoding: An Overview of various techniques and research issues. IEEE Trans. Multimed. 7, 5.
[2]
Bai, B., Boulanger, P., and Harms, J. 2005. A multiview video transcoder. In Proceedings of the ACM Multimedia. 503--506.
[3]
Cha, J., Eid, M., and Saddik, A. 2009. Touchable 3D video system. ACM Trans. Multimed. Comput. Commun. Appl. 5, 4.
[4]
Cheung, G., Ortega, A., and Cheung, N.-M. 2011. Interactive streaming of stored multiview video using redundant frame structures. IEEE Trans. Image Process. 20, 3.
[5]
Fehn, C. 2004. Depth-image-based-rendering (DIBR), compression and transmission for a new approach on 3D-TV. In SPIE Stereoscopic Displays and Virtual Reality Systems.
[6]
Florêncio, D. and Zhang, C. 2009. Multiview video compression and streaming based on predicted viewer position. In Proceedings of ICASSP '09.
[7]
JVT-AA209. 2008. Joint Draft 7.0 on multiview video coding.
[8]
JVT-T207. 2006. Common Test Conditions for Multiview Video Coding. Klagenfurt, Austria.
[9]
Kurutepe, E., Civanlar, M. R., and Tekalp, A. M. 2007. Client-driven selective streaming of multiview video for interactive 3DTV. IEEE Trans. Circuite Syst. Video. Technol. 17.
[10]
Levoy, M. and Hanrahan, P. 1996. Light field rendering, In Proceedings of SIGGRAPH'96, ACM, pp. 31--42.
[11]
Liu, S. and Chen, C. W. 2009. Multiview video transcoding: From multiple views to single view. In Proceedings of the Picture Coding Symposium (PCS'09).
[12]
Liu, S. and Chen, C. W. 2010. 3D video transcoding for virtual views. In Proceedings of ACM Multimedia.
[13]
Liu, Y., Huang, Q., Ma, S., Zhao, D., and Gao, W. 2009. Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Signal Process. Image Commun. 24, 8.
[14]
Liu, S., Lai, P., Tian, D., and Chen, C. W. 2011. New depth coding techniques with utilization of corresponding video. IEEE Trans. Broadcas. 57, 2.
[15]
Lou, J.-G., Cai, H., and Li, J. 2007. Interactive multiview video delivery based on IP multicast. In Advances in Multimedia.
[16]
Maitre, M. and Do, M. N. 2008. Joint encoding of the depth image based representation using shape-adaptive wavelets. In Proceedings of ICIP. 1768--1771
[17]
Microsoft 3D Video Test Sequences (available online: http://research.microsoft.com/en-us/um/people/sbkang/3dvideodownload/)
[18]
MPEG Video and Requirements Subgroup. 2009. Applications and requirements on 3D video coding. Document w11061. MPEG.
[19]
Müller, K., Smolic, A., Dix, K., Merkle, P., Kauff, P., and Wiegand, T. 2008. Reliability-based generation and view synthesis in layered depth video. In Proceedings of the IEEE International Workshop on Multimedia Signal Processing.
[20]
Oh, H. and Ho, Y.-S. 2006. H.264-based depth map coding using motion information of correspondingtexture video. Adv. Image Video Tech. 4319.
[21]
Pan, Z., Ikuta, Y., Bandai, M., and Watanabe, T. 2011. User dependent scheme for multi-view video transmission. In Proceedings of ICC.
[22]
Shum, H.-Y., Chan, S.-C., and Kang, S. B. 2006. Image-Based Rendering. Springer.
[23]
Smolic, A., Müller, K., Dix, K., Merkle, P., Kauff, P., and Wiegand, T. 2008. Intermediate View Interpolation Based on Multiview Video Plus Depth for Advanced 3D Video Systems. In Proceedings of the IEEE International Conference on Image Processing.
[24]
Smolic, A., Müller, K., Merkele, P., Kauff, P., and Wiegand, T. 2009. An overview of available and emerging 3D video formats and depth enhanced stereo as efficient generic solution. In Proceedings of the Picture Coding Symposium.
[25]
Tang, X.-L., Dai, S.-K., and Cai, C.-H. 2010. An analysis of TZSearch algorithm. In Proceedings of ICGCS.
[26]
Tanimoto, M., Fuji, T., and Suzuki, K. 2009. View synthesis algorithm in view synthesis reference software 3.0 (VSRS3.0), Tech. rep. Document M16090, ISO/IEC JTC1/SC29/WG11.
[27]
Vetro, A., Matusik, W., Pfister, H., and Xin, J. 2004. Coding approaches for end-to-end 3D TV systems. Proceedings of the Picture Coding Symposium.
[28]
Yang, Z., Wu, W., Nahrstedt, K., Kurillo G., and Bajscy, R. 2010. Enabling Multi-Party 3D Tele-Immersive Environments with ViewCast. ACM Trans. Multimedia Comput. Commun. Appl. 6, 2.
[29]
Yang, Y., Yu, M., Jiang, G., and Peng, Z. 2007. A transmission and interaction oriented free-viewpoint video system. Int. J. Circ. Syst. Signal Process. 4, 1.
[30]
Zhang, C. and Florêncio, D. 2010. Joint tracking and multiview video compression. In Proceedings of VCIP.
[31]
Zitnick, L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23, 3, 600--608.

Cited By

View all
  • (2023)Deep Saliency Mapping for 3D Meshes and ApplicationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/355007319:2(1-22)Online publication date: 6-Feb-2023
  • (2015)QoE-aware video streaming for SVC over multiuser MIMO-OFDM systemsJournal of Visual Communication and Image Representation10.1016/j.jvcir.2014.10.01126:C(24-36)Online publication date: 1-Jan-2015
  • (2014)Cloud Mobile MediaIEEE Transactions on Multimedia10.1109/TMM.2014.231559616:4(885-902)Online publication date: 1-Jun-2014

Index Terms

  1. A novel 3D video transcoding scheme for adaptive 3D video transmission to heterogeneous terminals

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Multimedia Computing, Communications, and Applications
    ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 8, Issue 3s
    Special section of best papers of ACM multimedia 2011, and special section on 3D mobile multimedia
    September 2012
    173 pages
    ISSN:1551-6857
    EISSN:1551-6865
    DOI:10.1145/2348816
    Issue’s Table of Contents
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 16 October 2012
    Accepted: 01 June 2012
    Revised: 01 April 2012
    Received: 01 January 2010
    Published in TOMM Volume 8, Issue 3s

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. 3D video
    2. Multiview video
    3. adaptive 3D video
    4. transcoding

    Qualifiers

    • Research-article
    • Research
    • Refereed

    Funding Sources

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)1
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 01 Mar 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Deep Saliency Mapping for 3D Meshes and ApplicationsACM Transactions on Multimedia Computing, Communications, and Applications10.1145/355007319:2(1-22)Online publication date: 6-Feb-2023
    • (2015)QoE-aware video streaming for SVC over multiuser MIMO-OFDM systemsJournal of Visual Communication and Image Representation10.1016/j.jvcir.2014.10.01126:C(24-36)Online publication date: 1-Jan-2015
    • (2014)Cloud Mobile MediaIEEE Transactions on Multimedia10.1109/TMM.2014.231559616:4(885-902)Online publication date: 1-Jun-2014

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media