Skip to main content
Log in

Encoder-independent decoder-dependent depth-assisted error concealment algorithm for wireless 3D video communication

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Three-Dimensional Multi-View Video (3D MVV) contains diverse video streams taken by different cameras around an object. Thence, it is an imperative assignment to fulfill efficient compression to attain future resource bonds whilst preserving a decisive reception MVV quality. The extensive 3D MVV encoding and transmission over mobile or Internet are vulnerable to packet losses on account of the existence of severe channel faults and restricted bandwidth. In this work, we propose a new Encoder-Independent Decoder-Dependent Depth-Assisted Error Concealment (EIDD-DAEC) algorithm. It invests the depth correlations between the temporally, spatially, and inter-view adjoining Macro-Blocks (MBs) to conceal the erroneous streams. At the encoder, the existing inter-view, temporal, and spatial matching are exploited to efficiently compress the 3D MVV streams and to estimate the Disparity Vectors (DVs) and Motion Vectors (MVs). At the decoder, the gathered MVs and DVs from the received coded streams are used to calculate additional depth-assisted MVs and DVs, which are afterwards combined with the collected candidate texture color MVs and DVs groups for concealing the lost MBs of inter- and intra-encoded frames. Finally, the optimum DVs and MVs concealment candidates are selected by the Directional Interpolation Error Concealment Algorithm (DIECA) and Decoder Motion Vector Estimation Algorithm (DMVEA), respectively. Experimental results on several standardized 3D MVV sequences verified the efficacy of the proposed EIDD-DAEC algorithm by achieving ameliorated efficacious objective and subjective results without generating and transporting depth maps at the encoder. The proposed work achieves high 3D MVV quality performance with an improved average Peak Signal-to-Noise Ratio (PSNR) gain by up to 0.95 ~ 2.70 dBs compared to the state-of-the-art error concealment algorithms, which do not employ depth-assisted correlations at different Quantization Parameters (QPs) and Packet Loss Rates (PLRs) of 40%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

References

  1. Abreu A, Frossard P, Pereira F (2015) Optimizing multiview video plus depth prediction structures for interactive multiview video streaming. IEEE J Select Topics Signal Process 9(3):487–500

    Article  Google Scholar 

  2. Assunçao P, Marcelino S, Soares S, Faria S (2016) Spatial error concealment for intra-coded depth maps in multiview video-plus-depth. Multimedia Tools and Applications, 1-24

  3. Cagri O, Erhan E, Janko C, Ahmet K (2016) Adaptive delivery of immersive 3D multi-view video over the internet. Multimed Tools Appl 75(20):12431–12461

    Article  Google Scholar 

  4. Chung TY, Sanghoon S, Kim CS (2011) Frame loss concealment for stereoscopic video plus depth sequences. IEEE Trans Consum Electron 57(3):1336–1344

    Article  Google Scholar 

  5. Ebdelli M, Le-Meur O, Guillemot C (2015) Video inpainting with short-term windows: application to object removal and error concealment. IEEE Trans Image Process 24(10):3034–3047

    Article  MathSciNet  Google Scholar 

  6. El Shafai W, Hrušovský B, El-Khamy M, El-Sharkawy M (2011) Joint space-time-view error concealment algorithms for 3D multi-view video. In: 18th IEEE Int. Conference on image processing (ICIP), pp 2201-2204

  7. El-Shafai W (2013) Optimized adaptive space-time-view multi-Dimentional error concealment for 3D multi-view video transmission. In: IEEE Saudi international Electronics, communications and photonics Conference (SIECPC), pp. 1-6

  8. El-Shafai W (2015) Pixel-level matching based multi-hypothesis error concealment modes for wireless 3D H.264/MVC communication. 3D Res 6(3):31

    Article  Google Scholar 

  9. El-Shafai W (2015) Joint adaptive pre-processing resilience and post-processing concealment schemes for 3D video transmission. 3D Res 6(1):1–13

    Article  Google Scholar 

  10. Gadgil N, Li H, Delp E.J (2015) Spatial subsampling-based multiple description video coding with adaptive temporal-spatial error concealment. In: IEEE Int conf on picture coding symposium (PCS) pp 90–94

  11. H.264/AVC codec. (2015) http://iphome.hhi.de/suehring/tml/: Last accessed on 28/09/2015

  12. Hewage CTER, Martini MG (2013) Quality of experience for 3D video streaming. IEEE Commun Mag 51(5):101–107

    Article  Google Scholar 

  13. Hong CS, Wang CC, Tai SC, Luo YC (2011) Object-based error concealment in 3D video. In: IEEE Int Conf on genetic and evolutionary computing (ICGEC) pp 5-8

  14. Huanqiang Z, Xiaolan W, Canhui C, Jing C, Yan Z (2014) Fast multiview video coding using adaptive prediction structure and hierarchical mode decision. IEEE Trans Circuit Syst Vid Technol 24(9):1566–1578

    Article  Google Scholar 

  15. Hwang M, Ko S (2008) Hybrid temporal error concealment methods for block-based compressed video transmission. IEEE Trans Broadcast 54(2):198–207

    Article  Google Scholar 

  16. ISO/IEC JTC1/SC29/WG11 (2006) Common test conditions for multiview Video coding. JVT-U207, Hangzhou, China

  17. Jun L, Yu Z, Xi Z, Jian S (2012) A dynamic hybrid UXP/ARQ method for scalable video transmission. In: 23rd IEEE Int Symposium on personal indoor and mobile radio communications (PIMRC), pp 2566-2571

  18. Khattak S, Maugey T, Hamzaoui R, Ahmad S, Frossard P (2016) Temporal and inter-view consistent error concealment technique for multiview plus depth video. IEEE Trans Circuit Syst Vid Technol 26(5):829–840

    Article  Google Scholar 

  19. Lie WN, Lee CM, Yeh CH, Gao ZW (2014) Motion vector recovery for video error concealment by using iterative dynamic-programming optimization. IEEE Trans Multimed 16(1):216–227

    Article  Google Scholar 

  20. Liu Z, Cheung G, Ji Y (2013) Optimizing distributed source coding for interactive multiview video streaming over lossy networks. IEEE Trans Circuit Syst Vid Technol 23(10):1781–1794

    Article  Google Scholar 

  21. Purica A, Mora E, Pesquet B, Cagnazzo M, Ionescu B (2016) Multiview plus depth video coding with temporal prediction view synthesis. IEEE Trans Circuit Syst Vid Technol 26(2):360–374

    Article  Google Scholar 

  22. Reference software for multiview video coding (mvc). (2015) http://wftp3.itu.int/av-arch/jvt-site/2009_01_Geneva/JVT-AD207.zip: Last accessed on 25/10/2015

  23. Salim O.H, Wei X, Leis J (2013) An efficient unequal error protection scheme for 3-D video transmission. In: IEEE Int wireless communications and networking conference (WCNC), pp 4077–4082

  24. Tai SC, Wang CC, Hong CS, Luo YC (2016) An effiicient full frame algorithm for object-based error concealment in 3D depth-based video. Multimed Tool Appl 75(16):9927–9947

    Article  Google Scholar 

  25. Wang H, Wang X (2016) Important macroblock distinction model for multi-view plus depth video transmission over error-prone network. Multimed Tool Appl:1–23

  26. Wen NL, Guan HL (2013) Error concealment for 3D video transmission. In: IEEE Int Symposium on Circuits and systems (ISCAS) pp 2856-2559

  27. Xiang X, Cheng Y (2014) A frame loss error concealment scheme for multi-view video coding. In: IEEE fourth Int Conf on instrumentation and measurement. Computer, Communication and Control (IMCCC), pp 857–861

    Google Scholar 

  28. Xiang W, Gao P, Peng Q (2015) Robust multiview three-dimensional video communications based on distributed video coding. IEEE SystJ PP(99):1–11

    Google Scholar 

  29. Yan B, Jie Z (2012) Efficient frame concealment for depth image-based 3-D video transmission. IEEE Trans Multimed 14(3):936–941

    Article  Google Scholar 

  30. Ying C, Vetro A (2014) Next generation 3D formats with depth map support. IEEE MultiMed 21(2):90–94

    Article  Google Scholar 

  31. Yongkai H, El-Hajjar M, Hanzo L (2013) Inter layer FEC aided unequal error protection for multilayer video transmission in mobile TV. IEEE Trans Circuit Syst Vid Technol 23(9):1622–1634

    Article  Google Scholar 

  32. Yunqiang L, Jin W, Huanhuan Z (2010) Depth image-based temporal error concealment for 3-D video transmission. IEEE Trans Circuit Syst Vid Technol 20(4):600–604

    Article  Google Scholar 

  33. Zhou Y, Xiang W, Wang G (2015) Frame loss concealment for multiview video transmission over wireless multimedia sensor networks. IEEE Sensors J 15(3):1892–1901

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to W. El-Shafai.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

El-Shafai, W., El-Rabaie, S., El-Halawany, M.M. et al. Encoder-independent decoder-dependent depth-assisted error concealment algorithm for wireless 3D video communication. Multimed Tools Appl 77, 13145–13172 (2018). https://doi.org/10.1007/s11042-017-4936-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-4936-y

Keywords

Navigation