Skip to main content
SpringerLink
Log in

Important macroblock distinction model for multi-view plus depth video transmission over error-prone network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Multi-view plus depth (MVD) video is an efficient three dimensional (3D) video representation format that allows the sender only transmit two pairs of texture and depth videos, arbitrary virtual view can be synthesized at the receiver. However, constrained by the limited-bandwidth, inevitable packet loss will induce transmission distortion, which will propagate to virtual views thereby affects user’s stereoscopic perception. In order to relieve the virtual view’s quality degradation induced by packet loss, an important macroblock (MB) distinction model for both texture and depth videos is proposed. MBs with low important level will be actively discarded once congestion occurs. The model includes two main parts: Firstly, by considering temporal and spatial correlation of the coding structure and the distortion diffusion due to lost packets, a transmission distortion model is proposed. Secondly, a gradient based synthesis distortion model is adopted to analyze the distortion induced by depth-error. Finally, a low-complexity important MB distinction model is proposed for MVD video transmission. Experiment results show that, compared with random packet loss condition, Peak Signal-to-Noise Ratio (PSNR) of the virtual view increase by up to 15.65 dB at 20% packet loss rate, both objective and subjective quality of the virtual view are close to error-free transmission.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alajel K, Xiang W, Wang Y (2012) Unequal error protection scheme based hierarchical 16-QAM for 3-D video transmission. IEEE Trans Consum Electron 58:731–738

    Article  Google Scholar 

  2. Baccaglini E, Tillo T, Olmo G (2011) Image and video transmission: a comparison study of using unequal loss protection and multiple description coding. Multimed Tools Appl 55(2):247–259

    Article  MATH  Google Scholar 

  3. Chen Z, Pahalawatta PV, Tourapis AM, Wu D (2012) Improved estimation of transmission distortion for error-resilient video coding. IEEE Trans Circuits Syst Video Technol 22:636–647

    Article  Google Scholar 

  4. Fang L, Cheung N-M, Tian D, Anthony V, Sun H, C AO (2014) An analytical model for synthesis distortion estimation in 3D video. IEEE Trans Image Process 23:185–199

    Article  MathSciNet  MATH  Google Scholar 

  5. Gao P, Xiang W (2014) Rate-distortion optimized mode switching for error-resilient multi-view video plus depth based 3-D video coding. IEEE Trans Multimed 16:1797–1808

    Article  Google Scholar 

  6. Heo YS, Lee KM, Lee SU (2011) Robust stereo matching using adaptive normalized cross-correlation. IEEE Trans Pattern Anal Mach Intell 33:807–822

    Article  Google Scholar 

  7. Hui Y, Yilin C, Junyan H, Fuzheng Y, Zhaoyang L (2011) Model-based joint bit allocation between texture videos and depth maps for 3-D video coding. IEEE Trans Circuits Syst Video Technol 21:485–497

    Article  Google Scholar 

  8. ISO/IEC MPEG, ITU-T VCEG (2012) Joint multi-view video coding model (JMVC 8.5)

  9. Kim W-S, Ortega A, Lai P, Tian D (2015) Depth map coding optimization using rendered view distortion for 3D video coding. IEEE Trans Image Process 24:3534–3545

    Article  MathSciNet  Google Scholar 

  10. Lambert P, De Neve W, Dhondt Y, Van de Walle R (2006) Flexible macroblock ordering in H.264/AVC. J Vis Commun Image Represent 17:358–375

    Article  Google Scholar 

  11. Li F, Liu G, He L (2010) Cross-layer scheduling for multiuser H.264 video transmission over wireless networks. IET Commun 4(8):1012–1025

    Article  Google Scholar 

  12. Li F, Zhang D, Wang L (2015) Packet importance based scheduling strategy for H.264 video transmission in wireless networks. Multimed Tools Appl 74:10259–10275

    Article  Google Scholar 

  13. Liu Q, Xin W, Giannakis GB (2006) A cross-layer scheduling algorithm with QoS support in wireless networks. IEEE Trans Veh Technol 55(3):839–847

    Article  Google Scholar 

  14. Liu Y, Liu J, Ci S, Ye Y (2013) Joint video/depth/FEC rate allocation with considering 3D visual saliency for scalable 3D video streaming. 2013 Visual Communications and Image Processing (VCIP), Kuching, pp. 1–6. doi:10.1109/VCIP.2013.6706339

  15. Loghman M, Kim J (2015) Segmentation-based view synthesis for multi-view video plus depth. Multimed Tools Appl 74(5):1611–1625

    Article  Google Scholar 

  16. Luo L, Jiang R, Tian X, Chen Y (2013) Rate-distortion based reference viewpoints selection for multi-view video plus depth coding. IEEE Trans Consum Electron 59:657–665

    Article  Google Scholar 

  17. Macchiavello B, Dorea C, Hung EM, Cheung G, Tan W-T (2014) Loss-resilient coding of texture and depth for free-viewpoint video conferencing. IEEE Trans Multimedia 16:711–725

    Article  Google Scholar 

  18. Moving Picture Experts Group, Geneva (2009), View synthesis software manual release 3.5 (VSRS 3.5),” Tech. Rep, ISO/IEC JTC1/SC29/WG11 MPEG, Sep. 2009

  19. Nagoya university ftv test sequences (2010) [Online]. Available: http://www.tanimoto.nuee.nagoya-u.ac.jp/

  20. Oh BT, Lee J, Park D (2011) Depth map coding based on synthesized view distortion function. IEEE journal of selected topics in. Signal Process 5:1344–1352

    Google Scholar 

  21. Pahalawatta PV, Pappas TN, Berry R, Katsaggelos AK (2007) Content-aware resource allocation and packet scheduling for video transmission over wireless networks. IEEE J Sel Areas Commun 25(4):749–759

    Article  Google Scholar 

  22. Philipp M, Aljoscha S, Karsten M, Thomas W (2007) Multi-view video plus depth representation and coding. IEEE International Conference on Image Processing. IEEE, pp 201–204

  23. Shen C, van der Schaar M (2008) Optimal resource allocation for multimedia applications over multi-access fading channels. IEEE Trans Wirel Commun 7(9):3546–3557

  24. Stockhammer T, Bystrom M (2004) H.264/AVC data partitioning for mobile video communication. International Conference on image processing, 2004 ICIP ‘04 545–548

  25. Wang X, Wang T, Hu B, Jiang G, Zhang L (2014) An important frame distinction model of stereoscopic video based on content. J Multimed 9(8):985–9941

    Google Scholar 

  26. Wu J, Shang Y, Qiao X, Cheng B, Chen J (2013) Robust bandwidth aggregation for real-time video delivery in integrated heterogeneous wireless networks. Multimed Tools Appl 74:4117–4138

  27. Xiao J, Hannuksela MM, Tillo T, Gabbouj M, Zhu C, Zhao Y (2015) Scalable bit allocation between texture and depth views for 3-D video streaming over heterogeneous networks. IEEE Trans Circuits Syst Video Technol 25:139–152

    Article  Google Scholar 

  28. Xu L, Au OC, Sun W, Fang L, Zou F, Li J (2015) Stereo matching with optimal local adaptive radiometric compensation. IEEE Sig Process Lett 22:131–135

    Article  Google Scholar 

  29. Yang H, Rose K (2010a) Optimizing motion compensated prediction for error resilient video coding. IEEE Trans Image Process 19:108–118

    Article  MathSciNet  MATH  Google Scholar 

  30. Zhou Y, Hou C, Pan R, Yuan Z, Yang L (2010) Distortion analysis and error concealment for multi-view video transmission. IEEE International symposium on broadband multimedia systems and broadcasting (BMSB). IEEE, pp 1–5

  31. Yuan H, Liu J, Xu H, Li Z, Liu W (2012) Coding distortion elimination of virtual view synthesis for 3D video system: theoretical analyses and implementation. IEEE Trans Broadcast 58:558–568

    Article  Google Scholar 

  32. Yuan H, Kwong S, Liu J, Sun J (2014) A novel distortion model and Lagrangian multiplier for depth maps coding. IEEE Trans Circuits Syst Video Technol 24:443–451

    Article  Google Scholar 

  33. Zhang R, Regunathan SL, Rose K (2000) Video coding with optimal inter/intra-mode switching for packet loss resilience. IEEE J Sel Areas Commun 18:966–976

    Article  Google Scholar 

  34. Zhang C, Yin Z, Florencio D (2009) Improving depth perception with motion parallax and its application in teleconferencing. 2009 I.E. Int Workshop Multimed Signal Process 278:287

    Google Scholar 

  35. Zhang T, Fan X, Zhao D, Gao W (2012) New distortion model for depth coding in 3DVC. Vis Commun Image Process:1–6. doi:10.1109/VCIP.2012.6410848

  36. Zhou Y, Hou C, Xiang W, Wu F (2011) Channel distortion modeling for multi-view video transmission over packet-switched networks. IEEE Trans Circuits Syst Video Technol 21:1679–1692

    Article  Google Scholar 

  37. Zhou Y, Xiang W, Wang G (2015) Frame loss concealment for multiview video transmission over wireless multimedia sensor networks. IEEE Sensors J 15:1892–1901

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Electronics and Information Engineering, Tongji University, Shanghai, China

    Hanzhang Wang

  2. Faculty of Information Science and Engineering, Ningbo University, Ningbo, China

    Hanzhang Wang & Xiaodong Wang

Authors
  1. Hanzhang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaodong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hanzhang Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, H., Wang, X. Important macroblock distinction model for multi-view plus depth video transmission over error-prone network. Multimed Tools Appl 76, 26745–26767 (2017). https://doi.org/10.1007/s11042-016-4204-6

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-4204-6

Keywords

Search

Navigation