Skip to main content
Springer Nature Link
Log in

Adaptive Learning Based View Synthesis Prediction for Multi-View Video Coding

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

In the applications of Free View TV, pre-estimated depth information is available to synthesize the intermediate views as well as to assist multi-view video coding. Existing view synthesis prediction schemes generate virtual view picture only from interview pictures. However, there are many types of signal mismatches caused by depth errors, camera heterogeneity or illumination difference across views and these mismatches decrease the prediction capability of virtual view picture. In this paper, we propose an adaptive learning based view synthesis prediction algorithm to enhance the prediction capability of virtual view picture. This algorithm integrates least square prediction with backward warping to synthesize the virtual view picture, which not only utilizes the adjacent views information but also the temporal decoded information to adaptively learn the prediction coefficients. Experiments show that the proposed method reduces the bitrates by up to 18 % relative to the multi-view video coding standard, and about 11 % relative to the conventional view synthesis prediction method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9

Similar content being viewed by others

References

  1. Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG. (2003). Draft ITU-T recommendation and final draft international standard of joint video specification (ITU-T Rec. H.264-ISO/IEC14496-10 AVC) JVT-G050.

  2. ISO/IEC JTC1/SC29/WG11. (2008). Text of ISO/IEC 14496-10:200X/FDAM 1 multi-view video coding. Doc. N9978.

  3. ISO/IEC MPEG & ITU-T VCEG. (2007). Multi-view Video plus Depth (MVD) format for advanced 3D video system. Doc. JVT-W100.

  4. Martinian, E., Behrens, A., Xin, J., & Vetro, A. (2006). View synthesis for multi-view video compression. In: Proc. Picture Coding Symposium PCS, Beijing, China.

  5. Na, S.-T., Oh, K.-J., & Ho, Y.-S. (2008). Joint coding of multi-view video and corresponding depth map. In: Proc. International Conference on Image Processing ICIP, San Diego, USA, 2468-2471.

  6. Yea, S., & Vetro, A. (2009). View synthesis prediction for multi0view video coding. Signal Processing: Image Communication, 24(1), 89–100.

    Google Scholar 

  7. Iyer, K. N., Maiti, K., Navathe, B., Kannan, H., & Sharma A. (2010). Multiview video coding using depth based 3D warping. In: Proc. International Conference on Multimedia and Expo ICME, Singapore, 1108–1113.

  8. Shimizu, S., Kimata, H., Sugimoto, S., & Matsuura N. (2011). Decoder side macroblock information derivation for efficient multi-view video plus depth map coding. In: Proc. 3DTV Conference, Turkey.

  9. Kim, W.-S., & Ortega A. (2009). Depth map distortion analysis for view rendering and depth coding. In: Proc. International Conference on Image Processing ICIP, Cairo, Egypt, 721–724.

  10. Merkle, P., Muller, K., Smolic, A., & Wiegand T. (2006). Efficient compression of multi-view video exploiting inter-view dependencies based on H.264/MPEG4-AVC. In: Proc. IEEE International Conference on Multimedia and Exposition, July.

  11. Yamamoto, K., Kitahara, M., & Kimata, H. (2007). Multi-view video coding using view interpolation and color correction. IEEE Transaction on Circuits and Systems for Video Technology, 17(1), 1436–1449.

    Article  Google Scholar 

  12. Hur, J.-H., Cho, S., & Lee, Y.-L. (2007). Adaptive local illumination change compensation method for H.264/AVC-based multi-view video coding. IEEE Transaction on Circuits and Systems for Video Technology, 17(11), 1496–1505.

    Article  Google Scholar 

  13. Telea, A. (2004). An image in-painting technique based on the fast marching method. Journal of Graphics Tools, 9(1), 25–36.

    Article  Google Scholar 

  14. Mori, Y., Fukushima, N., Fujii, T., & Tanimoto, M. (2008). View generation with 3D warping using depth information for FTV. In: Proc. 3DTV Conference, May, 229–232.

  15. Chen, Y., Pandit, P., Yea, S., & Lim, C.S. (2009). Draft reference software for MVC. Joint VideoTeam (JVT) of ISO/IEC MPEG & ITU-T VCEG, ISO/IEC JTC1/SC29/WG11 and ITU-T SG16 Q.6,Doc. JVT-AE207, London.

  16. Zitnick, C. L., Kang, S. B., Uyttendaele, M., & Szeliski, R. (2004). High-quality video view interpolation using a layered representation. In: Proc. ACM SIGGRAPH, 600–608.

  17. GIST 3D Video Sequences, Available at: ftp://203.253.128.142.

  18. ETRI/MPEG Korea Forum 3D Video Sequences, Available at: ftp://203.253.128.142.

  19. Bjøntegaard, G. (2001). Calculation of average PSNR differences between RD-curves. VCEG Doc. VCEG-M33, April.

Download references

Acknowledgments

The research was supported by the major national science and technology special projects (2010ZX03004-003-03, 2010ZX03004-001-03), the National Basic Research Program of China (973 Program) (2009CB320906), the National Natural Science Foundation of China (60832002, 60970160, 61070080, 61003184, 61271256), 2011 Academic Scholarship for Doctoral Candidates of Wuhan University.

Author information

Authors and Affiliations

  1. National Engineering Research Center for Multimedia Software, School of Computer, Wuhan University, Wuhan, 430079, China

    Jinhui Hu, Ruimin Hu, Zhongyuan Wang, Ge Gao, Mang Duan & Yan Gong

Authors
  1. Jinhui Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ruimin Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zhongyuan Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Ge Gao
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Mang Duan
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Yan Gong
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ruimin Hu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Hu, J., Hu, R., Wang, Z. et al. Adaptive Learning Based View Synthesis Prediction for Multi-View Video Coding. J Sign Process Syst 74, 115–126 (2014). https://doi.org/10.1007/s11265-013-0741-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-013-0741-7

Keywords