Skip to main content
Springer Nature Link
Log in

Spatial error concealment for intra-coded depth maps in multiview video-plus-depth

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Transmission errors or packet loss in depth maps have great impact on the decoding quality and view synthesis of 3D and multiview video. Thus efficient methods to recover corrupted depth data are critical functions for accurate view rendering. This paper proposes an error concealment method for intra-coded depth maps, based on spatial intra and inter-view methods, which exploit neighbouring data of depth and colour images received error-free. A novel three-stage processing algorithm is devised to reconstruct sharp depth transitions (i.e. lost depth contours), using a disparity map and geometric interpolation based on parametric Bézier curves. The simulation results obtained from different views of various MVD sequences, for different packetisation modes and a wide range of packet loss rates (PLR), show that the proposed method consistently leads to quality improvement of synthesised images in comparison with reference methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Ali A, Karim H, Arif N, Sali A (2010) Depth image-based spatial error concealment for 3-D video transmission. In: 2010 IEEE student conference on research and development (SCOReD), pp 421– 425

  2. Canny J (1986) A computational approach to edge dxetection. IEEE Trans Pattern Anal Mach Intell PAMI-8(6):679–698

    Article  Google Scholar 

  3. Chen Y, Wang Y, Ugur K, Hannuksela M, Lainema J, Gabbouj M (2008) The emerging MVC standard for 3D video services. EURASIP J. ASP 2009:8:1–8:13

    Google Scholar 

  4. Chen Z, Zhou P, He Y (2002) Fast Integer Pel and Fractional Pel Motion Estimation for JVT. Documents JVT-F017, JVT 6th Meeting, Awaji Island, Japan

  5. Chung B, Yim C (2014) Hybrid error concealment method combining exemplar-based image inpainting and spatial interpolation. Signal Process Image Commun 29(10):1121–1137

    Article  Google Scholar 

  6. Chung T, Sull S, Kim C (2010) Frame loss concealment for stereoscopic video based on inter-view similarity of motion and intensity difference. In: 2010 17th IEEE international conference on image processing (ICIP), pp 441–444

  7. Chung T, Sull S, Kim C (2011) Frame loss concealment for stereoscopic video plus depth sequences. IEEE Trans Consum Electron 57(3):1336–1344

    Article  Google Scholar 

  8. Ekmekcioglu E, Velisavljevic V, Worrall S (2011) Content Adaptive Enhancement of Multi-View Depth Maps for free Viewpoint Video. IEEE J Sel Top Signal Process 5 (2):352–361

    Article  Google Scholar 

  9. Hasan M, Arnold J, Frater M (2014) Subjective evaluation and statistical analysis for improved frame-loss error concealment of 3d videos. In: visual communications and image processing conference, 2014 IEEE, pp 181–184

  10. Hewage C, Martini M (2011) Joint error concealment method for backward compatible 3D video transmission. In: 2011 IEEE 73rd vehicular technology conference (VTC Spring), pp 1–5

  11. Hewage C, Martini M (2012) Edge-based reduced-reference quality metric for 3-D video compression and transmission. IEEE J Sel Top Signal Process 6(5):471–482

    Article  Google Scholar 

  12. Hewage C, Worrall S, Dogan S, Kondoz A (2008) Frame concealment algorithm for stereoscopic video using motion vector sharing. In: 2008 IEEE international conference on multimedia and expo, pp 485–488

  13. Hewage C, Worrall S, Dogan S, Kondoz A (2008) A novel frame concealment method for depth maps using corresponding colour motion vectors. In: 3DTV conference: the true vision - capture, transmission and display of 3D video, pp 149–152

  14. http://iphome.hhi.de/suehring/tml/: SuehringJ M17H.264/AVC, Last accessed on 02/09/2015

  15. http://opencv.itseez.com/: OpenCV-2.1.0, Last accessed on 02/09/2015

  16. http://vision.middlebury.edu/stereo/eval/: Middlebury Stereo Evaluation, Last accessed on 02/09/2015

  17. Hughes JF, van Dam A, McGuire M, Sklar DF, Foley JD, Feiner SK, Akeley K (1990) Computer graphics, principles and practice. Addison Wesley

    Google Scholar 

  18. Jain R, Ramakrishnan KK, Chiu DM (2010) Definition of a general and intuitive loss model for packet networks and its implementation in the netem module in the linux kernel. Technical Report, Digital Equipment Corporation

  19. Khattak S, Maugey T, Hamzaoui R, Ahmad S, Frossard P (2015) Temporal and inter-view consistent error concealment technique for multiview plus depth video. IEEE Trans Circ Syst Video Technol PP(99):1–1

    Google Scholar 

  20. Kim Y, Kim J, Sohn K (2007) Fast disparity and motion estimation for multi-view video coding. IEEE Transactions on Consumer Electronics 53(2):712–719

    Article  Google Scholar 

  21. Kordelas A, Politis I, Dagiuklas T (2015) Transport analysis and quality evaluation of mvc video streaming. In: multimedia tools and applications, pp 1–26

  22. Kosov S, Thormahlen T, Seidel H (2009) Accurate real-time disparity estimation with variational methods. In: advances in visual computing, lecture notes in computer science, vol 5875. Springer, Berlin Heidelberg, pp 796–807

  23. Lie WN, Lin GH (2015) Error concealment for the transmission of H.264/AVC-compressed 3D video in color plus depth format. J Visual Commun Image Represent 32:237–245

    Article  Google Scholar 

  24. Lin TL, Chang TE, Huang GX, Chou CC, Thakur US (2014) Improved interview video error concealment on whole frame packet loss. J Vis Commun Image Represent 25(8):1811–1822

    Article  Google Scholar 

  25. Liu S, Chen Y, Wang Y, Gabbouj M, Hannuksela M, Li H (2008) Frame loss error concealment for multiview video coding. In: international symposium on circuits and systems, 2008. ISCAS 2008. IEEE, pp 3470–3473

  26. Liu X, Peng Q, Fan X (2011) Frame loss concealment for multi-view video plus depth. In: 2011 IEEE 15th international symposium on consumer electronics (ISCE), pp 208–211

  27. Liu Y, Wang J, Zhang H (2010) Depth image-based temporal error concealment for 3-D video transmission. IEEE Trans Circ Syst Video Technol 20(4):600–604

    Article  Google Scholar 

  28. Marcelino S, Assuncao P, De Faria SMM, Soares S (2011) Error recovery of image-based depth maps using bezier curve fitting. In: IEEE international conference on image processing (ICIP) pp 2293– 2296

  29. Marcelino S, Faria S, Pepion R, Callet P, Soares S, Assuncao P (2015) Quality evaluation of depth map error concealment using a perceptually-aware objective metric. In: 3DTV-Conference: the true vision - capture, transmission and display of 3D video (3DTV-CON), pp 1–4

  30. Merkle P, Morvan Y, Smolic A, Farin D, Muller K, de With P, Wiegand T (2008) The effect of depth compression on multiview rendering quality. In: 3DTV conference: the true vision - capture, transmission and display of 3D video, pp 245–248

  31. Micallef B, Debono C (2010) Error concealment techniques for multi-view video. In: Wireless days (WD), 2010 IFIP, pp 1–5

  32. Micallef B, Debono C, Farrugia R (2011) Performance of enhanced error concealment techniques in multi-view video coding systems. In: 2011 18th international conference on systems, signals and image processing (IWSSIP), pp 1–4

  33. Ostermann J, Bormans J, List P, Marpe D, Narroschke M, Pereira F, Stockhammer T, Wedi T (2004) Video coding with H.264/AVC: tools, performance, and complexity. IEEE Circ Syst Mag 4(1):7–28

    Article  Google Scholar 

  34. Rares A, Reinders M, Biemond J (2005) Edge-based image restoration. IEEE Trans Image Process 14(10):1454–1468

    Article  MATH  Google Scholar 

  35. Saponara S, Blanch C, Denolf K, Bormans J (2003) The JVT advanced video coding standard: Complexity and performance analysis on a tool-by-tool basis. In: packet video workshop (PVW03)

  36. Soares L, Pereira F (2004) Spatial shape error concealment for object-based image and video coding. IEEE Trans Image Process 13(4):586–599

    Article  Google Scholar 

  37. Stankiewicz O, Wegner K, Domanski M (2010) Error concealment for MVC and 3D video coding. In: picture coding symposium (PCS), pp 498–501

  38. Tai SC, Wang CC, Hong CS, Luo YC (2015) An effiicient full frame algorithm for object-based error concealment in 3d depth-based video. Multimedia tools and applications pp 1–21

  39. Tanimoto M, Fujii T, Suzuki K (2009) View synthesis algorithm in view synthesis reference software 3.5 (VSRS3.5) Document M16090, ISO/IEC JTC1/SC29/WG11 (MPEG)

  40. Vetro A, Wiegand T, Sullivan G (2011) Overview of the stereo and multiview video coding extensions of the H.264/MPEG-4 AVC Standard. Proc IEEE 99(4):626–642

    Article  Google Scholar 

  41. Wiegand T, Sullivan G, Bjontegaard G, Luthra A (2003) Overview of the H.264/AVC video coding standard. IEEE Trans CSVT 13(7):560–576

    Google Scholar 

  42. Xiang X, Zhao D, Ma S, Gao W (2011) Auto-regressive model based error concealment scheme for stereoscopic video coding. In: 2011 IEEE international conference on acoustics, speech and signal processing (ICASSP) pp. 849–852

  43. Yan B A novel H.264 based motion vector recovery method for 3D video transmission. IEEE Transactions on Consumer Electronics 53(4):1546–1552

  44. Yan B, Zhou J (2012) Efficient frame concealment for depth image based 3D video transmission. IEEE Trans Multimed PP(99):1

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Instituto de Telecomunicações, Leiria, Portugal

    Pedro A. Amado Assunçao, Sylvain Marcelino & Sérgio M. M. de Faria

  2. Vila Real, Portugal

    Sylvain Marcelino & Salviano Soares

  3. IEETA, UA Campus, Aveiro, Portugal

    Salviano Soares

  4. Instituto Politécnico de Leiria/ESTG, Leiria, Portugal

    Pedro A. Amado Assunçao & Sérgio M. M. de Faria

Authors
  1. Pedro A. Amado Assunçao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sylvain Marcelino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Salviano Soares
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sérgio M. M. de Faria
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pedro A. Amado Assunçao.

Additional information

This work was supported by the Fundação para a Ciência e Tecnologia, Portugal under PhD Grant SFRH/BD/64988/2009 and Project PTDC/EEA-TEL 114487/2009.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amado Assunçao, P.A., Marcelino, S., Soares, S. et al. Spatial error concealment for intra-coded depth maps in multiview video-plus-depth. Multimed Tools Appl 76, 13835–13858 (2017). https://doi.org/10.1007/s11042-016-3766-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-016-3766-7

Keywords

Search

Navigation