Skip to main content
SpringerLink
Log in

User-perceived quality aware adaptive streaming of 3D multi-view video plus depth over the internet

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Video streaming is a foremost and growing contributor in the ever increasing Internet traffic. Since last two decades, due to the enhancement in cameras and image processing technology, we have seen a shift towards multi-view plus depth (MVD) technology from traditional 2D and 3D video technology. This growth comes with deep changes in the Internet bandwidth, video coding and network technologies, which smoothed the mode for delivery of MVD content to end-users over the Internet. Since, MVD contains large amounts of data than single view video, it requires more bandwidth. It is a challenging task for network service provider to deliver such views with the best user’s Quality of Experience(QoE) in dynamic network condition. Also, Internet is known to be prone to packet loss, bandwidth variation, delay and network congestion, which may prevent video packets from being delivered on time. Besides that, different capabilities of end user’s devices in terms of computing power, display, and access link capacity are other challenges. As consequences, the viewing experiences of 3D videos may well degrade, if the quality-aware adaptation techniques are not deployed. In this article, our work concentrates to present a comprehensive analysis of a dynamic network environment for streaming of 3D MVD over Internet (HTTP). We analyzed the effect of different adaptation of decision strategies and formulated a new quality-aware adaptation technique. The proposed technique is promoting from layer based video coding in terms of transmitted views scalability. The results of MVD streaming experiment, using the proposed approach have shown that the video quality of perceptual 3D improves significantly, as an effect of proposed quality aware adaptation even in adverse network conditions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. (MPEG) IJSW (2010) Dynamic adaptive streaming over http. w11578,CD 23001–6, w11578, CD 23001–6. ISO/IEC JTC 1/SC 29/WG11

  2. Apple Developer (2016) Apple HTTP live streaming. https://developer.apple.com/streaming/

  3. Benzie P et al (2007) A survey of 3DTV displays: techniques and technologies. IEEE Trans Circuits Syst Video Technol 17(11):1647–1658

    Article  Google Scholar 

  4. Bosc E et al (2013) A study of depth/texture bit-rate allocation in multi-view video plus depth compression. annals of telecommunications-annales des télécommunications 68(11–12):615–625

    Article  Google Scholar 

  5. Buchowicz A (2013) Video coding and transmission standards for 3D television — a survey. Opto-Electron Rev 21:39

    Article  Google Scholar 

  6. Chakareski J (2013) Adaptive multiview video streaming: challenges and opportunities. IEEE Commun Mag 51(5):94–100

    Article  Google Scholar 

  7. De Simone F, Dufaux F (2013) Comparison of DASH adaptation strategies based on bitrate and quality signalling. In: Multimedia Signal Processing (MMSP), 2013 I.E. 15th international workshop on. IEEE

  8. H.-H.-I. Fruanhofer (2013) HEVC 3D extension test model (3DV HTM) version 11.0. Available from: https://hevc.hhi.fraunhofer.de/svn/svn_3DVCSoftware/tags/HTM-11.0/

  9. Gao Y et al (2017) Event classification in microblogs via social tracking. ACM Trans Intell Syst Technol (TIST) 8(3):35

    Google Scholar 

  10. Ho YS, Lee EK, Lee C (2008) Multiview video test sequence and camera parameters. Tech. Rep. MPEG2008/M15419 ISO/IEC JTC1/SC29/WG11. Archamps, France

  11. Internet Information Service (2014)

  12. ISO/IEC JTC1/SC29/WG11 (2005) Report of the subjective quality evaluation for MVC call for evidence. Tech. Rep. MPEG2005/N6999, Hong Kong, China

    Google Scholar 

  13. ITU-R Recommendation (2012) ITU-R BT.500–13, Methodology for the subjective assessment of the quality of television pictures. Tech. Rep

  14. Jacobson V (1998) Congestion avoidance and control. In: Proceeding of SIGCOMM ‘88. ACM SIGCOMM Computer Communication Review 18(4):314–329

  15. Kauff P et al (2007) Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability. Signal Process Image Commun 22(2):217–234

    Article  Google Scholar 

  16. Kuschnig R, Kofler I, Hellwagner H (2011) Evaluation of HTTP-based request-response streams for internet video streaming. In: Proceedings of the second annual ACM conference on Multimedia systems. ACM

  17. Li B, Li H, Li L, Zhang J (2012) Rate control by R-lambda model for HEVC. In: JCTVC-K0103, JCTVC of ISO/IEC and ITU-T, 11th Meeting, Shanghai, China

  18. Miller K et al (2012) Adaptation algorithm for adaptive streaming over HTTP. In: Packet Video Workshop (PV), 2012 19th International. IEEE

  19. Müller K et al (2013) 3D high-efficiency video coding for multi-view video and depth data. IEEE Trans Image Process 22(9):3366–3378

    Article  MathSciNet  MATH  Google Scholar 

  20. Ohm J-R et al (2012) Comparison of the coding efficiency of video coding standards—including high efficiency video coding (HEVC). IEEE Trans Circuits Syst Video Technol 22(12):1669–1684

    Article  Google Scholar 

  21. Ozcinar C et al (2016) Adaptive delivery of immersive 3D multi-view video over the internet. Multimed Tools Appl 75(20):12431–12461

    Article  Google Scholar 

  22. Ozcinar C, Ekmekcioglu E, Kondoz A (2016) Quality-aware adaptive delivery of multi-view video. In: Acoustics, Speech and Signal Processing (ICASSP), 2016 I.E. International Conference on. IEEE

  23. Oztas B et al (2014) A rate adaptation approach for streaming multiview plus depth content. in Computing, Networking and Communications (ICNC), 2014 International Conference on. IEEE

  24. Psannis KE, Hadjinicolaou MG, Krikelis A (2006) MPEG-2 streaming of full interactive content. IEEE Trans Circuits Syst Video Technol 16(2):280–285

    Article  Google Scholar 

  25. Rizzo L (1997) Dummynet: a simple approach to the evaluation of network protocols. ACM SIGCOMM Comput Commun Rev 27(1):31–41

    Article  Google Scholar 

  26. Roodaki, H., M.R. Hashemi, and S. Shirmohammadi, (2012) A new methodology to derive objective quality assessment metrics for scalable multiview 3D video coding. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), 2012. 8(3s): p. 44

  27. Savas SS, Tekalp AM, Gurler CG (2011) Adaptive multi-view video streaming over P2P networks considering quality of experience. In: Proceedings of the 2011 ACM workshop on social and behavioural networked media access. ACM

  28. Schierl T et al (2012) System layer integration of high efficiency video coding. IEEE Trans Circuits Syst Video Technol 22(12):1871–1884

    Article  Google Scholar 

  29. Seufert M et al (2015) A survey on quality of experience of HTTP adaptive streaming. IEEE Commun Surv Tutorials 17(1):469–492

    Article  Google Scholar 

  30. Shimizu S et al (2007) View scalable multiview video coding using 3-d warping with depth map. IEEE Trans Circuits Syst Video Technol 17(11):1485–1495

    Article  Google Scholar 

  31. Smolic A et al (2007) Coding Algorithms for 3DTV—A Survey. IEEE Trans Circuits Syst Video Technol 17(11):1606–1621

    Article  MathSciNet  Google Scholar 

  32. Smolic A et al (2008) Intermediate view interpolation based on multiview video plus depth for advanced 3D video systems. In: Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on. IEEE

  33. Sodagar I (2011) The MPEG-DASH standard for multimedia streaming over the internet. IEEE MultiMedia 18(4):62–67

    Article  Google Scholar 

  34. Sripanidkulchai K, Maggs B, Zhang H (2004) An analysis of live streaming workloads on the internet. In Proceedings of the 4th ACM SIGCOMM conference on internet measurement. ACM

  35. Stockhammer T (2011) Dynamic adaptive streaming over HTTP--: standards and design principles. In: Proceedings of the second annual ACM conference on Multimedia systems. ACM

  36. Sullivan GJ et al (2012) Overview of the high efficiency video coding (HEVC) standard. IEEE Trans Circuits Syst Video Technol 22(12):1649–1668

    Article  Google Scholar 

  37. Tanimoto M (2009) Overview of FTV (free-viewpoint television). In: Proceeding of 2009 I.E. International Conference on Multimedia and Expo, New York, pp 1552–1523

  38. Tanimoto M (2010) Overview of Free-viewpoint TV (FTV). In: Javidi B, Fournel T (eds) Information optics and photonics. Springer, New York. https://doi.org/10.1007/978-1-4419-7380-1_9

  39. Tanimoto M et al (2013) Proposal on a new activity for the third phase of FTV. In: the 105th meeting of MPEG

  40. Thang TC et al (2014) An evaluation of bitrate adaptation methods for HTTP live streaming. IEEE J Sel Areas Commun 32(4):693–705

    Article  Google Scholar 

  41. The official Microsoft IIS site (2016) Microsoft smooth-streaming. Microsoft Corporation 2016; Available from: https://www.iis.net/downloads/microsoft/smooth-streaming

  42. Toni L et al (2014) Optimal set of video representations in adaptive streaming. In: proceedings of the 5th ACM Multimedia Systems Conference. ACM

  43. Vetro A, Wiegand T, Sullivan GJ (2011) Overview of the stereo and multiview video coding extensions of the H. 264/MPEG-4 AVC standard. Proc IEEE 99(4):626–642

    Article  Google Scholar 

  44. Wiegand T et al (2003) Overview of the H. 264/AVC video coding standard. IEEE Trans Circuits Syst Video Technol 13(7):560–576

    Article  Google Scholar 

  45. Zhao S et al (2015) Strategy for dynamic 3D depth data matching towards robust action retrieval. Neurocomputing 151:533–543

    Article  Google Scholar 

  46. Zhao S et al (2015) View-based 3D object retrieval via multi-modal graph learning. Signal Process 112:110–118

    Article  Google Scholar 

  47. Zhao S, Yao H, Jiang X (2015) Predicting continuous probability distribution of image emotions in valence-arousal space. In: Proceedings of the 23rd ACM international conference on Multimedia. ACM

  48. Zhao S et al (2016) Predicting personalized emotion perceptions of social images. In: Proceedings of the 2016 ACM on Multimedia Conference. ACM

  49. Zhao S et al (2017) Continuous probability distribution prediction of image emotions via multitask shared sparse regression. IEEE Trans Multimedia 19(3):632–645

    Article  Google Scholar 

  50. Zhao S et al (2017) Approximating discrete probability distribution of image emotions by multi-modal features fusion. Transfer 1000:1

    Google Scholar 

  51. Zhao S, Gao Y, Ding G, Chua TS (2017) Real-time multimedia social event detection in microblog. IEEE Trans on Cybernetics 99:1–14

  52. Zhou C et al (2012) A control-theoretic approach to rate adaptation for dynamic HTTP streaming. In: Visual Communications and Image Processing (VCIP), 2012 IEEE. IEEE

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Harbin Institute of Technology, Harbin, China

    Nabin Kumar Karn, Hongli Zhang & Feng Jiang

Authors
  1. Nabin Kumar Karn
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongli Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feng Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nabin Kumar Karn.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Karn, N.K., Zhang, H. & Jiang, F. User-perceived quality aware adaptive streaming of 3D multi-view video plus depth over the internet. Multimed Tools Appl 77, 22965–22983 (2018). https://doi.org/10.1007/s11042-018-5744-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-018-5744-8

Keywords

Search

Navigation