Skip to main content
SpringerLink
Log in

Dynamic bitrate allocation of interactive real-time streamed multi-view video with view-switch prediction

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

In multi-view video (MVV), the real-world scene is usually captured by more than two cameras positioned in an array. A viewer can consume MVV using either a non-interactive or an interactive transmission method. In the context of interactive MVV streaming, view switching may cause a long delay due to the frequent requests by the viewer. In this paper, we consider the use case of real-time interactive MVV (IMVV) streaming, where the view switching delay problem has a significant user experience impact. Our proposed method compress and send all the captured views using a dynamic bitrate allocation method. Also, a novel prediction algorithm has been used to choose possible views that the user might switch to. The predicted view switching is mapped to a hidden Markov model, and the transition probability is solved using Zipf distribution. The experimental results of the proposed method show a superior performance on an objective metric and view-switching delay for better viewing quality over the existing method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Ozcinar, C., Ekmekcioglu, E., Kondoz, A.: Quality-aware adaptive delivery of multi-view video. In: 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Shanghai, Mar 2016 (accepted)

  2. Kuhn, A., Hirschmüller, H., Scharstein, D., Mayer, H.: A TV prior for high-quality scalable multi-view stereo reconstruction. Int. J. Comput. Vis. 1–16 (2016). doi:10.1007/s11263-016-0946-x

  3. Ozcinar, C., Ekmekcioglu, E., Kondoz, A.: Adaptive 3D multi-view video streaming over P2P networks. In: 2014 IEEE International Conference on Image Processing (ICIP), Paris, Oct 2014, pp. 2462–2466

  4. Lafruit, G., Wegner, K., Tanimoto, M.: Final Draft Call for Evidence on FTV. ISO/IEC JTC1/SC29/WG11/, Warsaw, Technical Report MPEG2015, June 2015

  5. Ozcinar, C., Ekmekcioglu, E., Kondoz, A.: HTTP adaptive multiview video streaming. In: Connected Media in the Future Internet Era, pp. 191–217. Springer, Berlin (2017)

  6. Yoon, S.-U., Lee, E.-K., Kim, S.-Y., Ho, Y.-S.: A framework for representation and processing of multi-view video using the concept of layered depth image. J. VLSI Signal Process. Syst. Signal Image Video Technol. 46(2–3), 87–102 (2007)

    Article  Google Scholar 

  7. Xu, W., Zou, J., Xiong, H.: Interactive multiview video scheduling through bargaining. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 3590–3594 (2015)

  8. Ozcinar, C., Ekmekcioglu, E., Ćalić, J., Kondoz, A.: Adaptive delivery of immersive 3D multi-view video over the internet. Multimed. Tools Appl. 75(20), 12 431–12 461 (2016)

    Article  Google Scholar 

  9. Cheung, G., Ortega, A., Cheung, N.-M.: Interactive streaming of stored multiview video using redundant frame structures. IEEE Trans. Image Process. 20(3), 744–761 (2011)

    Article  MathSciNet  Google Scholar 

  10. Tanimoto, M., Tehrani, M.P., Fujii, T., Yendo, T.: Free-viewpoint TV. IEEE Signal Process. Mag. 28(1), 67–76 (2011)

    Article  Google Scholar 

  11. Kurutepe, E., Civanlar, M.R., Tekalp, A.M.: Client-driven selective streaming of multiview video for interactive 3DTV. IEEE Trans. Circuits Syst. Video Technol. 17(11), 1558–1565 (2007)

    Article  Google Scholar 

  12. Scandarolli, T., de Queiroz, R.L., Florencio, D.A.: Attention-weighted rate allocation in free-viewpoint television. IEEE Signal Process. Lett. 20(4), 359–362 (2013)

    Article  Google Scholar 

  13. Chen, Y.-C., Yang, D.-N., Liao, W.: Efficient multi-view 3D video multicast with depth image-based rendering in LTE networks. In: 2013 IEEE Global Communications Conference (GLOBECOM), pp. 4427–4433 (2013)

  14. Fehn, C.: Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. In: International Society for Optics and Photonics Electronic Imaging 2004, pp. 93–104 (2004)

  15. Dorea, C., de Queiroz, R.L.: General rate-allocation in free-viewpoint television. In: 2014 IEEE International Conference on Image Processing (ICIP), pp. 145–149 (2014)

  16. Song, Y., Ho, Y.-S.: Unified depth intra coding for 3D video extension of HEVC. Signal Image Video Process. 8(6), 1031–1037 (2014)

    Article  Google Scholar 

  17. Xiu, X., Cheung, G., Liang, J.: Frame structure optimization for interactive multiview video streaming with bounded network delay. In: 2011 18th IEEE International Conference on Image Processing (ICIP), pp. 593–596 (2011)

  18. Zhang, B., Liu, Z., Chan, S.-H.G., Cheung, G.: Collaborative wireless freeview video streaming with network coding. IEEE Trans. Multimed. 18(3), 521–536 (2016)

    Article  Google Scholar 

  19. Xiu, X., Cheung, G., Liang, J.: Delay-cognizant interactive streaming of multiview video with free viewpoint synthesis. IEEE Trans. Multimed. 14(4), 1109–1126 (2012)

    Article  Google Scholar 

  20. Zhang, C., Florêncio, D.: Joint tracking and multiview video compression. In: Visual Communications and Image Processing 2010, International Society for Optics and Photonics, p. 77 440P (2010)

  21. Kim, I.-K., McCann, K., Sugimoto, K., Bross, B., Han, W.-J.: High efficiency video coding (HEVC) test model 10 (HM10) encoder description. ISO/IEC JTC1/SC29/WG11, Geneva, Technical Report N12242, Jan 2013

  22. Walther, D., Koch, C.: Modeling attention to salient proto-objects. Neural Netw. 19(9), 1395–1407 (2006)

    Article  MATH  Google Scholar 

  23. Viola, P., Jones, M.J.: Robust real-time face detection. Int. J. Comput. Vis. 57(2), 137–154 (2004)

    Article  Google Scholar 

  24. Baum, L.E., Petrie, T.: Statistical inference for probabilistic functions of finite state Markov chains. Ann. Math. Stat. 37, 1554–1563 (1966)

    Article  MathSciNet  MATH  Google Scholar 

  25. Zink, M., Suh, K., Gu, Y., Kurose, J.: Characteristics of youtube network traffic at a campus network–measurements, models, and implications. Comput. Netw. 53(4), 501–514 (2009). doi:10.1016/j.comnet.2008.09.022

    Article  Google Scholar 

  26. Itu.int, H.265.2(10/14) Reference software for ITU-T H.265 high efficiency video coding (2014) (online). http://www.itu.int/rec/T-REC-H.265.2

  27. De Boor, C., De Boor, C., De Boor, C., De Boor, C.: A Practical Guide to Splines, vol. 27. Springer, New York (1978)

    Book  MATH  Google Scholar 

  28. D. S. of Nagoya University, Nagoya University, Japan, http://www.tanimoto.nuee.nagoya-u.ac.jp/MPEG-FTVProject.html (Jan 2017) (online). http://www.tanimoto.nuee.nagoya-u.ac.jp/MPEG-FTVProject.html

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Statistics, Trinity College Dublin, Dublin 2, Ireland

    Cagri Ozcinar

  2. iCV Research Group, Institute of Technology, University of Tartu, 50411, Tartu, Estonia

    Gholamreza Anbarjafari

  3. Department of Electrical and Electronic Engineering, Hasan Kalyoncu University, Gaziantep, Turkey

    Gholamreza Anbarjafari

Authors
  1. Cagri Ozcinar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gholamreza Anbarjafari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gholamreza Anbarjafari.

Additional information

This work has been partially supported by Estonian Research Council Grant (PUT638), the Estonian Centre of Excellence in IT (EXCITE) funded by the European Regional Development Fund and the European Network on Integrating Vision and Language (iV&L Net) ICT COST Action IC1307. The authors also gratefully acknowledge the support of NVIDIA Corporation with the donation of the Titan X Pascal GPU.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ozcinar, C., Anbarjafari, G. Dynamic bitrate allocation of interactive real-time streamed multi-view video with view-switch prediction. SIViP 11, 1279–1285 (2017). https://doi.org/10.1007/s11760-017-1085-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-017-1085-8

Keywords

Search

Navigation