Abstract
Three-Dimensional Multi-View Video (3D MVV) contains diverse video streams taken by different cameras around an object. Thence, it is an imperative assignment to fulfill efficient compression to attain future resource bonds whilst preserving a decisive reception MVV quality. The extensive 3D MVV encoding and transmission over mobile or Internet are vulnerable to packet losses on account of the existence of severe channel faults and restricted bandwidth. In this work, we propose a new Encoder-Independent Decoder-Dependent Depth-Assisted Error Concealment (EIDD-DAEC) algorithm. It invests the depth correlations between the temporally, spatially, and inter-view adjoining Macro-Blocks (MBs) to conceal the erroneous streams. At the encoder, the existing inter-view, temporal, and spatial matching are exploited to efficiently compress the 3D MVV streams and to estimate the Disparity Vectors (DVs) and Motion Vectors (MVs). At the decoder, the gathered MVs and DVs from the received coded streams are used to calculate additional depth-assisted MVs and DVs, which are afterwards combined with the collected candidate texture color MVs and DVs groups for concealing the lost MBs of inter- and intra-encoded frames. Finally, the optimum DVs and MVs concealment candidates are selected by the Directional Interpolation Error Concealment Algorithm (DIECA) and Decoder Motion Vector Estimation Algorithm (DMVEA), respectively. Experimental results on several standardized 3D MVV sequences verified the efficacy of the proposed EIDD-DAEC algorithm by achieving ameliorated efficacious objective and subjective results without generating and transporting depth maps at the encoder. The proposed work achieves high 3D MVV quality performance with an improved average Peak Signal-to-Noise Ratio (PSNR) gain by up to 0.95 ~ 2.70 dBs compared to the state-of-the-art error concealment algorithms, which do not employ depth-assisted correlations at different Quantization Parameters (QPs) and Packet Loss Rates (PLRs) of 40%.
Similar content being viewed by others
References
Abreu A, Frossard P, Pereira F (2015) Optimizing multiview video plus depth prediction structures for interactive multiview video streaming. IEEE J Select Topics Signal Process 9(3):487–500
Assunçao P, Marcelino S, Soares S, Faria S (2016) Spatial error concealment for intra-coded depth maps in multiview video-plus-depth. Multimedia Tools and Applications, 1-24
Cagri O, Erhan E, Janko C, Ahmet K (2016) Adaptive delivery of immersive 3D multi-view video over the internet. Multimed Tools Appl 75(20):12431–12461
Chung TY, Sanghoon S, Kim CS (2011) Frame loss concealment for stereoscopic video plus depth sequences. IEEE Trans Consum Electron 57(3):1336–1344
Ebdelli M, Le-Meur O, Guillemot C (2015) Video inpainting with short-term windows: application to object removal and error concealment. IEEE Trans Image Process 24(10):3034–3047
El Shafai W, Hrušovský B, El-Khamy M, El-Sharkawy M (2011) Joint space-time-view error concealment algorithms for 3D multi-view video. In: 18th IEEE Int. Conference on image processing (ICIP), pp 2201-2204
El-Shafai W (2013) Optimized adaptive space-time-view multi-Dimentional error concealment for 3D multi-view video transmission. In: IEEE Saudi international Electronics, communications and photonics Conference (SIECPC), pp. 1-6
El-Shafai W (2015) Pixel-level matching based multi-hypothesis error concealment modes for wireless 3D H.264/MVC communication. 3D Res 6(3):31
El-Shafai W (2015) Joint adaptive pre-processing resilience and post-processing concealment schemes for 3D video transmission. 3D Res 6(1):1–13
Gadgil N, Li H, Delp E.J (2015) Spatial subsampling-based multiple description video coding with adaptive temporal-spatial error concealment. In: IEEE Int conf on picture coding symposium (PCS) pp 90–94
H.264/AVC codec. (2015) http://iphome.hhi.de/suehring/tml/: Last accessed on 28/09/2015
Hewage CTER, Martini MG (2013) Quality of experience for 3D video streaming. IEEE Commun Mag 51(5):101–107
Hong CS, Wang CC, Tai SC, Luo YC (2011) Object-based error concealment in 3D video. In: IEEE Int Conf on genetic and evolutionary computing (ICGEC) pp 5-8
Huanqiang Z, Xiaolan W, Canhui C, Jing C, Yan Z (2014) Fast multiview video coding using adaptive prediction structure and hierarchical mode decision. IEEE Trans Circuit Syst Vid Technol 24(9):1566–1578
Hwang M, Ko S (2008) Hybrid temporal error concealment methods for block-based compressed video transmission. IEEE Trans Broadcast 54(2):198–207
ISO/IEC JTC1/SC29/WG11 (2006) Common test conditions for multiview Video coding. JVT-U207, Hangzhou, China
Jun L, Yu Z, Xi Z, Jian S (2012) A dynamic hybrid UXP/ARQ method for scalable video transmission. In: 23rd IEEE Int Symposium on personal indoor and mobile radio communications (PIMRC), pp 2566-2571
Khattak S, Maugey T, Hamzaoui R, Ahmad S, Frossard P (2016) Temporal and inter-view consistent error concealment technique for multiview plus depth video. IEEE Trans Circuit Syst Vid Technol 26(5):829–840
Lie WN, Lee CM, Yeh CH, Gao ZW (2014) Motion vector recovery for video error concealment by using iterative dynamic-programming optimization. IEEE Trans Multimed 16(1):216–227
Liu Z, Cheung G, Ji Y (2013) Optimizing distributed source coding for interactive multiview video streaming over lossy networks. IEEE Trans Circuit Syst Vid Technol 23(10):1781–1794
Purica A, Mora E, Pesquet B, Cagnazzo M, Ionescu B (2016) Multiview plus depth video coding with temporal prediction view synthesis. IEEE Trans Circuit Syst Vid Technol 26(2):360–374
Reference software for multiview video coding (mvc). (2015) http://wftp3.itu.int/av-arch/jvt-site/2009_01_Geneva/JVT-AD207.zip: Last accessed on 25/10/2015
Salim O.H, Wei X, Leis J (2013) An efficient unequal error protection scheme for 3-D video transmission. In: IEEE Int wireless communications and networking conference (WCNC), pp 4077–4082
Tai SC, Wang CC, Hong CS, Luo YC (2016) An effiicient full frame algorithm for object-based error concealment in 3D depth-based video. Multimed Tool Appl 75(16):9927–9947
Wang H, Wang X (2016) Important macroblock distinction model for multi-view plus depth video transmission over error-prone network. Multimed Tool Appl:1–23
Wen NL, Guan HL (2013) Error concealment for 3D video transmission. In: IEEE Int Symposium on Circuits and systems (ISCAS) pp 2856-2559
Xiang X, Cheng Y (2014) A frame loss error concealment scheme for multi-view video coding. In: IEEE fourth Int Conf on instrumentation and measurement. Computer, Communication and Control (IMCCC), pp 857–861
Xiang W, Gao P, Peng Q (2015) Robust multiview three-dimensional video communications based on distributed video coding. IEEE SystJ PP(99):1–11
Yan B, Jie Z (2012) Efficient frame concealment for depth image-based 3-D video transmission. IEEE Trans Multimed 14(3):936–941
Ying C, Vetro A (2014) Next generation 3D formats with depth map support. IEEE MultiMed 21(2):90–94
Yongkai H, El-Hajjar M, Hanzo L (2013) Inter layer FEC aided unequal error protection for multilayer video transmission in mobile TV. IEEE Trans Circuit Syst Vid Technol 23(9):1622–1634
Yunqiang L, Jin W, Huanhuan Z (2010) Depth image-based temporal error concealment for 3-D video transmission. IEEE Trans Circuit Syst Vid Technol 20(4):600–604
Zhou Y, Xiang W, Wang G (2015) Frame loss concealment for multiview video transmission over wireless multimedia sensor networks. IEEE Sensors J 15(3):1892–1901
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
El-Shafai, W., El-Rabaie, S., El-Halawany, M.M. et al. Encoder-independent decoder-dependent depth-assisted error concealment algorithm for wireless 3D video communication. Multimed Tools Appl 77, 13145–13172 (2018). https://doi.org/10.1007/s11042-017-4936-y
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-4936-y