Skip to main content
SpringerLink
Log in

A Rate–Distortion Optimized Error-Resilient Algorithm for Multi-view Video Coding

  • Published:
Circuits, Systems, and Signal Processing Aims and scope Submit manuscript

Abstract

In this paper, a new rate–distortion (R–D) optimized error-resilient scheme is proposed to improve error resilience for multi-view video transmission over lossy networks. Based on the study on the characteristics of multi-view video coding and the propagating behavior of channel errors, a recursive model to estimate the end-to-end distortion is first developed, in which the channel-induced distortion takes into consideration both motion and disparity compensation prediction. Then, based on the estimated distortion and the derived total bit rate, R–D optimized motion and disparity estimation are employed to find the reliable motion vector and disparity vector to achieve the optimum trade-off between prediction efficiency and error robustness. Finally, R–D optimization is applied again to perform coding mode decision, which explicitly considers intra, inter and inter-view prediction modes. Extensive experimental results demonstrate significant performance gains can be achieved for multi-view video communications against transmission errors.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Z. Chen, P.V. Pahalawatta, A. Michael Tourapis, D. Wu, Improved estimation of transmission distortion for error-resilient video coding. IEEE Trans. Circuits Syst. Video Technol. 22(4), 636–647 (2012)

    Article  Google Scholar 

  2. G. Cote, S. Shirani, F. Kossentini, Optimal mode selection and synchronization for robust video communications over error-prone networks. IEEE J. Sel. Areas Commun. 18(6), 952–965 (2000)

    Article  Google Scholar 

  3. Description of core experiments in MVC, ISO/IEC JTC1/SC29/WG11, MPEG2006/W7798, Apr. 2006

  4. M.B. Dissanayake, D.V.S.D. Silva, S.T. Worrall, and W.A.C. Fernando, Error resilient technique for multi-view coding using redundant disparity vectors, in Proceedings of IEEE International Conference Multimedia and Expo (ICME), (2010), pp. 1712–1717

  5. P. Gao, W. Xiang, Rate–distortion optimized mode switching for error-resilient multi-view video plus depth based 3-D video coding. IEEE Trans. Multimed. 16(7), 1797–1808 (2014)

    Article  Google Scholar 

  6. Y. Guo, Y. Chen, Y. Wang, H. Li, M.M. Hannuksela, M. Gabbouj, Error resilient coding and error concealment in scalable video coding. IEEE Trans. Circuits Syst. Video Technol. 19(6), 781–795 (2009)

    Article  Google Scholar 

  7. Z. He, J. Cai, C. Wen Chen, Joint source channel rate–distortion analysis for adaptive mode selection and rate control in wireless video coding. IEEE Trans. Circuits Syst. Video Technol. 12(6), 511–523 (2002)

    Article  Google Scholar 

  8. Joint Video Team of ISO/IEC MPEG and ITU-T VCEG, WD1 Reference Software for MVC, ISO/IECJTC1/SC29/WG11 and ITU-T SG16 Q.6 JVT-AA212, Apr. 2008

  9. A. Kubota, A. Smolic, M. Magnor, M. Tanimoto, T. Chen, C. Zhang, Multiview imaging and 3DTV. IEEE Signal Process. Mag. 24(6), 10–21 (2007)

    Article  Google Scholar 

  10. S. Ma, W. Gao, Y. Lu, Rate-distortion analysis for H.264/AVC video and its application to rate control. IEEE Trans. Circuits Syst. Video Technol. 15(12), 1533–1544 (2005)

    Article  Google Scholar 

  11. P. Merkle, A. Smolic, K. Muller, T. Wiegand, Efficient prediction structures for multiview video coding. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1461–1473 (2007)

    Article  Google Scholar 

  12. K. Song, T. Chung, Y. Oh, C.S. Kim, Error concealment of multi-view video sequences using inter-view and intra-view correlations. J. Vis. Commun. Image Represent. 20(4), 281–292 (2009)

    Article  Google Scholar 

  13. G.J. Sullivan, T. Wiegand, Rate–distortion optimization for video compression. IEEE Signal Process. Mag. 15(6), 74–90 (1998)

    Article  Google Scholar 

  14. A. Vetro, T. Wiegand, G.J. Sullivan, Overview of the stereo and multi-view video coding extensions of the H.264/MPEG-4 AVC standard. Proceedings of the IEEE 99(4), 626–642 (Apr. 2011)

  15. S. Wan, Y. Chang, and F. Yang, Frame-layer rate control for JVT video coding using improved quadratic rate distortion model, in Proceedings of Visual Communication and Image Processing, (2005), pp. 1053–1060

  16. S. Wan, E. Izquierdo, Rate–distortion optimized motion-compensated prediction for packet loss resilient video coding. IEEE Trans. Image Process. 16(5), 1327–1338 (2007)

    Article  MathSciNet  Google Scholar 

  17. Z. Wang, A. Bovik, H. Sheikh, E. Simoncelli, Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  18. Y. Wang, Z. Wu, J.M. Boyce, Modeling of transmission-loss induced distortion in decoded video. IEEE Trans. Circuits Syst. Video Technol. 16(6), 716–732 (2006)

    Article  Google Scholar 

  19. H. Wang, S. Kwong, Rate–distortion optimization of rate control for H.264 with adaptive initial quantization parameter determination. IEEE Trans. Circuits Syst. Video Technol. 18(1), 140–144 (2008)

    Article  Google Scholar 

  20. T. Wiegand, B. Girod, Lagrange multiplier selection in hybrid video coder control, in Proceedings IEEE International Conference on Image Processing (ICIP), (2001), pp. 542–545

  21. T. Wiegand, N. Farber, K. Stuhlmuller, B. Girod, Error-resilient video transmission using long-term memory motion-compensated prediction. IEEE J. Sel. Areas Commun. 18(6), 1050–1062 (2000)

    Article  Google Scholar 

  22. X. Xiang, D. Zhao, Q. Wang, S. Ma, W. Gao, A joint encoder–decoder error control framework for stereoscopic video coding. J. Vis. Commun. Image Represent. 21(8), 975–985 (2010)

    Article  Google Scholar 

  23. H. Yang, K. Rose, Advance in recursive per-pixel end-to-end distortion estimation for robust video coding in H.264/AVC. IEEE Trans. Circuits Syst. Video Technol. 17(7), 845–856 (2007)

    Article  Google Scholar 

  24. H. Yang, K. Rose, Optimizing motion compensated prediction for error resilient video coding. IEEE Trans. Image Process. 19(1), 108–118 (2010)

    Article  MathSciNet  Google Scholar 

  25. D. Zhang, J. Liang, and I. Singh, End-to-end distortion estimation for H.264 with unconstrained intra prediction, in Proceedings of IEEE International Conference on Image Processing (ICIP), (2012), pp. 697–700

  26. R. Zhang, S.L. Regunathan, K. Rose, Video coding with optimal inter/intra-mode switching for packet loss resilience. IEEE J. Sel. Areas Commun. 18(6), 966–976 (2000)

    Article  Google Scholar 

  27. Y. Zhang, W. Gao, Y. Lu, Q. Huang, D. Zhao, Joint source-channel rate-distortion optimization for H.264 video coding over error-prone networks. IEEE Trans. Multimedia 9(3), 445–454 (2007)

    Article  Google Scholar 

  28. Y. Zhou, C. Hou, W. Xiang, F. Wu, Channel distortion modeling for multi-view video transmission over packet-switched networks. IEEE Trans. Circuits Syst. Video Technol. 21(11), 1679–1692 (2011)

    Article  Google Scholar 

Download references

Acknowledgments

The work was supported in part by a Smart Futures Fellowship funded by the Queensland Government of Commonwealth Australia and the Postgraduate Scholarship Program of China Scholarship Council (2013-2015).

Author information

Authors and Affiliations

  1. School of Mechanical and Electrical Engineering, University of Southern Queensland, Toowoomba, QLD, 4350, Australia

    Pan Gao, Wei Xiang & John Billingsley

  2. School of Information Science and Technology, Southwest Jiaotong University, Chengdu, 610031, Sichuan, China

    Pan Gao & Qiang Peng

Authors
  1. Pan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Xiang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiang Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. John Billingsley
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pan Gao.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gao, P., Xiang, W., Peng, Q. et al. A Rate–Distortion Optimized Error-Resilient Algorithm for Multi-view Video Coding. Circuits Syst Signal Process 35, 301–323 (2016). https://doi.org/10.1007/s00034-015-0065-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00034-015-0065-x

Keywords

Search

Navigation