Abstract
As the representative and most widely used content form of Virtual Reality (VR) application, omnidirectional videos provide immersive experience for users with 360-degree scenes rendered. Since only part of the omnidirectional video can be viewed at a time due to human’s eye characteristics, field of view (FoV) based transmission has been proposed by ensuring high quality in the FoV while reducing the quality out of that to lower the amount of transmission data. In this case, transient content quality reduction will occur when the user’s FoV changes, which can be improved by predicting the FoV beforehand. In this paper, we propose a two-layer model for FoV prediction. The first layer detects the heat maps of content in offline process, while the second layer predicts the FoV of a specific user online during his/her viewing period. We utilize a LSTM model to calculate the viewing probability of each region given the results from the first layer, the user’s previous orientations and the navigation speed. In addition, we set up a correction model to check and correct the unreasonable results. The performance evaluation shows that our model obtains higher accuracy and less undulation compared with widely used approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Virtual and augmented reality: Understanding the race for the next computing platform. Technical report. The Goldman Sachs Group Inc. (2016)
Bao, Y., Wu, H., Zhang, T., Ramli, A.A., Liu, X.: Shooting a moving target: motion-prediction-based transmission for 360-degree videos. In: 2016 IEEE International Conference on Big Data (Big Data), pp. 1161–1170. IEEE (2016)
Corbillon, X., Simon, G., Devlic, A., Chakareski, J.: Viewport-adaptive navigable 360-degree video delivery. In: 2017 IEEE International Conference on Communications (ICC), pp. 1–7. IEEE (2017)
De Abreu, A., Ozcinar, C., Smolic, A.: Look around you: Saliency maps for omnidirectional images in vr applications. In: 2017 Ninth International Conference on Quality of Multimedia Experience (QoMEX), pp. 1–6. IEEE (2017)
Fan, C.L., Lee, J., Lo, W.C., Huang, C.Y., Chen, K.T., Hsu, C.H.: Fixation prediction for 360 video streaming in head-mounted virtual reality. In: Proceedings of the 27th Workshop on Network and Operating Systems Support for Digital Audio and Video, pp. 67–72. ACM (2017)
Graf, M., Timmerer, C., Mueller, C.: Towards bandwidth efficient adaptive streaming of omnidirectional video over http: Design, implementation, and evaluation. In: Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 261-271. ACM (2017)
Hochreiter, Sepp, Schmidhuber, Jürgen: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Mohammad H., Swaminathan, V.: Adaptive 360 vr video streaming: Divide and conquer. In: 2016 IEEE International Symposium on Multimedia (ISM), pp. 107–110. IEEE (2016)
Hu, Y., Xie, S., Xu, Y., Sun, J.: Dynamic VR live streaming over MMT. In: 2017 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting (BMSB), pp. 1–4. IEEE (2017)
Lipton, Z.C., Berkowitz, J., Elkan, C.: A critical review of recurrent neural networks for sequence learning (2015). arXiv:1506.00019
Lo, W.C., Fan, C.L., Lee, J., Huang, C.Y., Chen, K.T., Hsu, C.H.: 360 video viewing dataset in head-mounted virtual reality. In: Proceedings of the 8th ACM on Multimedia Systems Conference, pp. 211–216 (2017)
Qian, F., Ji, L., Han, B., Gopalakrishnan, V.: Optimizing 360 video delivery over cellular networks. In: Proceedings of the 5th Workshop on All Things Cellular: Operations, Applications and Challenges, pp. 1–6 (2016)
Sitzmann, Vincent, Serrano, Ana, Pavel, Amy, Agrawala, Maneesh, Gutierrez, Diego, Masia, Belen, Wetzstein, Gordon: Saliency in vr: How do people explore virtual environments? IEEE Trans. Vis. Comput. Graph. 24(4), 1633–1642 (2018)
Wu, Z., Su, L., Huang, Q., Wu, B., Li, J., Li, G.: Video saliency prediction with optimized optical flow and gravity center bias. In: 2016 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2016)
Xie, S., Xu, Y., Qian, Q., Shen, Q. Ma, Z., Zhang, W.: Modeling the perceptual impact of viewport adaptation for immersive video. In: 2018 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 1–5. IEEE (2018)
Xu, M., Song, Y., Wang, J., Qiao, M., Huo, L., Wang, Z.: Modeling Attention in Panoramic Video: A Deep Reinforcement Learning Approach (2017)
Acknowledgements
This paper is supported in part by National Natural Science Foundation of China (61650101), Scientific Research Plan of the Science and Technology Commission of Shanghai Municipality (16511104203), in part by the 111 Program (B07022).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Li, Y., Xu, Y., Xie, S., Ma, L., Sun, J. (2019). Two-Layer FoV Prediction Model for Viewport Dependent Streaming of 360-Degree Videos. In: Liu, X., Cheng, D., Jinfeng, L. (eds) Communications and Networking. ChinaCom 2018. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 262. Springer, Cham. https://doi.org/10.1007/978-3-030-06161-6_49
Download citation
DOI: https://doi.org/10.1007/978-3-030-06161-6_49
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-06160-9
Online ISBN: 978-3-030-06161-6
eBook Packages: Computer ScienceComputer Science (R0)