Abstract
In the multi-view video plus depth 3D video coding, texture image and depth map are coded jointly. The texture image is utilized for displaying and synthesizing the virtual view as reference image. The depth map provides the scene geometry information and is utilized to synthesize the virtual view at the terminal through Depth-Image Based Rendering technique. The distortion of the compressed texture image and depth map will be propagated to the synthesized virtual view. Besides the coding efficiency of texture image and depth map, bit allocation between texture image and depth map also has a great effect on the synthesized virtual view quality. Several methods are proposed for bit allocation between texture image and depth map, but most of them attempt to allocate a fixed target bitrate based on virtual view distortion model to achieve optimal synthesized virtual view quality, and the modeling process brings extra complexity. In practical application, the video sequence has different contents and fixed bit ratio cannot achieve optimal performance. In this paper, we propose an adaptive bit allocation algorithm for 3D video coding. First, we present a model to estimate the synthesized virtual view distortion, and then adjust the bit ratio between adjacent views and between texture image and depth map at Group of Picture level based on the virtual view quality fluctuation. We adjust the bit ratio to achieve the optimal virtual view quality for different video contents. Experimental results demonstrate that the proposed algorithm can optimally allocate bits to achieve optimal virtual view quality under different target bitrates and for different video contents, and the computational complexity of the proposed algorithm is extremely low.
Similar content being viewed by others
References
3DV Sequences of Nagoya University. Nagoya University, Nagoya. http://www.tanimoto.nuee.nagoyau.ac.jp/mpeg/mpeg-ftv.html. Mar 2008
3DV Sequences of Poznan University. Poznan University, Poznan. ftp://multimedia.edu.pl/3DV/. Nov 2012
3DV Sequences of HHI. Fraunhofer Heinrich Hertz Institute, Berlin. ftp://ftp.hhi.de/HHIMPEG3DV. Sep 2013
3D-HTM reference software version 10.0rc1. https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-10.0rc1/
P. Benzie, J. Watson, P. Surman, I. Rakkolainen, K. Hopf, H. Urey, V. Sainov, C. von Kopylow, A survey of 3DTV displays: techniques and technologies. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1647–1658 (2007)
G. Bjontegaard, Calculation of average PSNR difference between RD-curves, in 13th VCEG-M33 Meeting 2001, Austin, TX, USA (2001)
E. Bosc, V. Jantet, M. Pressigout, L. Morin, C. Guillemot, Bit-rate allocation for multi-view video plus depth, in 3DTV Conference: The True Vision-Capture, Transmission and Display of 3D Video (3DTV-CON) (2011), pp. 1–4
Y. Chen, M.M. Hannuksela, T. Suzuki, S. Hattori, Overview of the MVC+D 3D video coding standard. J. Vis. Commun. Image Represent. 25(4), 679–688 (2014)
C. Fehn, Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV, in Proceedings of SPIE. International Society for Optics and Photonics Electronic Imaging (2004), pp. 93–104
C. Fehn, R. de la Barré, S. Pastoor, Interactive 3-DTV-concepts and key technologies. Proc. IEEE 94(3), 524–538 (2006)
M. M. Hannuksela, Y. Chen, T. Suzuki, J.R. Ohm, G.J. Sullivan, 3D-AVC Draft Text 8, document JCT-3V JCT3V-F1002, Nov 2013
P. Kauff, N. Atzpadin, C. Fehn, M. Müller, O. Schreer, A. Smolic, R. Tanger, Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Signal Process. Image Commun. 22(2), 217–234 (2007)
W.S. Kim, A. Ortega, P. Lai, D. Tian, C. Gomila, Depth map coding with distortion estimation of rendered view, in IS&T/SPIE Electronic Imaging. International Society for Optics and Photonics (2010), pp. 75430B–75430B-10
J. Konrad, M. Halle, 3-D displays and signal processing. IEEE Signal Process. Mag. 24(6), 97–111 (2007)
B. Li, H. Li, L. Li, J. Zhang, Rate control by R-lambda model for HEVC, in ITU-T SG16 Contribution, JCTVC-K0103, Shanghai (2012), pp. 1–5
Y. Liu, Q. Huang, S. Ma, D. Zhao, W. Gao, Joint video/depth rate allocation for 3D video coding based on view synthesis distortion model. Signal Process. Image Commun. 24(8), 666–681 (2009)
Y. Liu, Q. Huang, S. Ma, D. Zhao, W. Gao, S. Ci, H. Tang, A novel rate control technique for multiview video plus depth based 3D video coding. IEEE Trans. Broadcast. 57(2), 562–571 (2011)
P. Merkle, A. Smolic, K. Muller, T. Wiegand, Multi-view video plus depth representation and coding, in Proceedings of IEEE ICIP (2007), pp. 201–204
Y. Morvan, D. Farin, H.N. de With, Joint depth/texture bit-allocation for multi-view video compression, in Proceedings of Picture Coding Symposium (PCS 2007) (2007), pp. 265–268
Y.K. Park, K. Jung, Y. Oh, S. Lee, J.K. Kim, G. Lee, H. Lee, K. Yun, N. Hur, J. Kim, Depth-image-based rendering for 3DTV service over T-DMB. Signal Process. Image Commun. 24(1), 122–136 (2009)
D. Rusanovskyy, K. Mueller, A. Vetro, Common test conditions of 3DV core experiments, in Joint Collaborative Team on 3D Video Coding Extensions (JCT-3V 2013) Document JCT3V-E1100, 5th Meeting, Vienna, Austria (2013)
F. Shao, G. Jiang, W. Lin, M. Yu, Q. Dai, Joint bit allocation and rate control for coding multi-view video plus depth based 3D video. IEEE Trans. Multimed. 15(8), 1843–1854 (2013)
L. Shen, Z. Zhang, Z. Liu, Effective CU size decision for HEVC intra coding. IEEE Trans. Image Process. 23(10), 4232–4241 (2014)
L. Shen, Z. Zhang, Z. Liu, Adaptive inter-mode decision for HEVC jointly utilizing inter-level and spatiotemporal correlations. IEEE Trans. Circuits Syst. Video Technol. 24(10), 1709–1722 (2014)
G.J. Sullivan, J.R. Ohm, W.J. Han, T. Wiegand, Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1688 (2012)
G. Tech, K. Wegner, Y. Chen, S. Yea, 3D-HEVC Test Model 3, MPEG number m28377, Geneve, Switzerland, Jan 2013
Q. Wang, X. Ji, Q. Dai, N. Zhang, Free viewpoint video coding with rate-distortion analysis. IEEE Trans. Circuits Syst. Video Technol. 22(6), 875–889 (2012)
J. Xiao, M.M. Hannuksela, T. Tillo, C. Zhu, Y. Zhao, Scalable bit allocation between texture and depth views for 3-D video streaming over heterogeneous networks. IEEE Trans. Circuits Syst. Video Technol. 25(1), 139–152 (2015)
J. Xiao, T. Tillo, H. Yuan, Y. Zhao, Macroblock level bits allocation for depth maps in 3-D video coding. J. Signal Process. Syst. 74(1), 127–135 (2013)
X. Xu, L.M. Po, K.W. Cheung, K.H. Ng, K.M. Wong, C.W. Ting, A foreground biased depth map refinement method for DIBR view synthesis, in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2012), pp. 805–808
C. Yang, P. An, D. Liu, L. Shen, Virtual view distortion estimation for depth map coding, in IEEE Visual Communications and Image Processing (VCIP), Singapore, Dec 2015, pp. 1–4
H. Yuan, J. Liu, H. Xu, Z. Li, W. Liu, Coding distortion elimination of virtual view synthesis for 3D video system: theoretical analyses and implementation. IEEE Trans. Broadcast. 58(4), 558–568 (2012)
H. Yuan, Y. Chang, J. Huo, F. Yang, Z. Lu, Model-based joint bit allocation between texture videos and depth maps for 3-D video coding. IEEE Trans. Circuits Syst. Video Technol. 21(4), 485–497 (2011)
Y. Zhang, S. Kwong, L. Xu, S. Hu, C. Kuo, G. Jiang, Regional bit allocation and rate distortion optimization for multiview depth video coding with view synthesis distortion model. IEEE Trans. Image Process. 22(9), 3497–3512 (2013)
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China, under Grants U1301257, 61172096, 61571285, 61422111 and 61301112.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Yang, C., An, P., Shen, L. et al. Adaptive Bit Allocation for 3D Video Coding. Circuits Syst Signal Process 36, 2102–2124 (2017). https://doi.org/10.1007/s00034-016-0402-8
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00034-016-0402-8