Skip to main content
SpringerLink
Log in

Deep Reinforcement Learning Based Collaborative Mobile Edge Caching for Omnidirectional Video Streaming

  • Conference paper
  • First Online:
Wireless Algorithms, Systems, and Applications (WASA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12937))

Abstract

Omnidirectional virtual reality video streaming requires high data rate and low latency to provide immersive experiences. Caching video contents at collaborative wireless edge servers is a promising approach to enable omnidirectional video streaming. Given limited storage at edges and ultra-high rate of omnidirectional video streaming, collaborative caching is challenging. Although restricting caching to parts of field of views (FoVs) of omnidirectional videos can alleviate the problem, the FoV based collaborative caching is still difficult considering multiple video encoding versions, long-term benefits for multiple end users and possible FoV prediction error. In this paper, we propose a deep reinforcement learning (DRL) based scheme to learn a collaborative caching policy adaptively for tile-based omnidirectional video streaming, where watched tiles are cached with abundant storage space, and either discarded or replacing existing content when edge storages are full. The collaborative cache replacement is modeled as a Markov decision process (MDP). To address the problem of searching a large-scale action space in DRL, we designed an adaptive action-selection scheme utilizing FoV prediction results. Simulation results demonstrate the effectiveness of the proposed scheme.

This work is supported by National Natural Science Foundation of China under Grant No. 61801167, and the Fundamental Research Funds for the Central Universities under Grant No. B200202189.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Khurana, T.: Retail and AR and VR: 5G brings it home! (2019). https://blogs.cisco.com/sp/retail-and-ar-and-vr-5g-brings-it-home?dtid=osscdc000283

  2. Dai, J., Zhang, Z., Mao, S., Liu, D.: A view synthesis-based 360\(^{\circ }\) VR caching system over MEC-enabled C-RAN. IEEE Trans. Circuits Syst. Video Technol. 30, 3843–3855 (2019)

    Article  Google Scholar 

  3. Sukhmani, S., Sadeghi, M., Erol-Kantarci, M., El Saddik, A.: Edge caching and computing in 5G for mobile AR/VR and tactile internet. IEEE Multimedia 26(1), 21–30 (2019)

    Article  Google Scholar 

  4. Han, S., Su, H., Yang, C., Molisch, A.F.: Proactive edge caching for video on demand with quality adaptation. IEEE Trans. Wireless Commun. 19(1), 218–234 (2020)

    Article  Google Scholar 

  5. Wang, D., et al.: Adaptive wireless video streaming based on edge computing: opportunities and approaches. IEEE Trans. Serv. Comput. 12(5), 685–697 (2019). https://doi.org/10.1109/TSC.2018.2828426

    Article  Google Scholar 

  6. Tran, T.X., Hajisami, A., Pandey, P., Pompili, D.: Collaborative mobile edge computing in 5G networks: new paradigms, scenarios, and challenges. IEEE Commun. Mag. 55(4), 54–61 (2017). https://doi.org/10.1109/MCOM.2017.1600863

    Article  Google Scholar 

  7. Chakareski, J.: VR/AR immersive communication: caching, edge computing, and transmission trade-offs. In: Proceedings of the Workshop on Virtual Reality and Augmented Reality Network. VR/AR Network 2017, pp. 36–41 (2017). https://doi.org/10.1145/3097895.3097902

  8. Song, Y., Zhao, Y., Li, C.: A user behavior aware immersive video caching algorithm. In: Proceedings of the 11th International Conference on Wireless Communications and Signal Processing. WCSP 2019, pp. 1–6 (2019). https://doi.org/10.1109/WCSP.2019.8927955

  9. Sullivan, G.J., Ohm, J., Han, W., Wiegand, T.: Overview of the high efficiency video coding (HEVC) standard. IEEE Trans. Circuits Syst. Video Technol. 22(12), 1649–1668 (2012)

    Article  Google Scholar 

  10. Chakareski, J.: Viewport-adaptive scalable multi-user virtual reality mobile-edge streaming. IEEE Trans. Image Process. 29, 6330–6342 (2020). https://doi.org/10.1109/TIP.2020.2986547

    Article  Google Scholar 

  11. Maniotis, P., Bourtsoulatze, E., Thomos, N.: Tile-based joint caching and delivery of 360\(^{\circ }\) videos in heterogeneous networks. In: Proceedings of the 21st International Workshop on Multimedia Signal Processing. MMSP 2019, pp. 1–6 (2019)

    Google Scholar 

  12. He, D., Jiang, J., Westphal, C., Yang, G.: Efficient edge caching for high-quality 360-degree video delivery. In: Ro, Y.M., et al. (eds.) MMM 2020. LNCS, vol. 11962, pp. 39–51. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37734-2_4

    Chapter  Google Scholar 

  13. Yang, Y., Lee, J., Kim, N., Kim, K.: Social-viewport adaptive caching scheme with clustering for virtual reality streaming in an edge computing platform. Future Gener. Comput. Syst. 108, 424–431 (2020). https://doi.org/10.1016/j.future.2020.02.078

    Article  Google Scholar 

  14. Sun, L., Mao, Y., Zong, T., Liu, Y., Wang, Y.: Flocking-based live streaming of 360-degree video. In: Proceedings of the 11th ACM Multimedia Systems Conference. MMSys 2020, pp. 26–37 (2020). https://doi.org/10.1145/3339825.3391856

  15. Mahzari, A., Taghavi Nasrabadi, A., Samiei, A., Prakash, R.: FoV-aware edge caching for adaptive 360\(^{\circ }\) video streaming. In: Proceedings of the 26th ACM International Conference on Multimedia. MM 2018, pp. 173–181 (2018). https://doi.org/10.1145/3240508.3240680

  16. Li, D., et al.: Deep reinforcement learning for cooperative edge caching in future mobile networks. In: Proceedings of the 2019 IEEE Wireless Communications and Networking Conference, pp. 1–6 (2019). https://doi.org/10.1109/WCNC.2019.8885516

  17. Yang, Z., Liu, Y., Chen, Y., Tyson, G.: Deep reinforcement learning in cache-aided MEC networks. In: Proceedings of the 2019 IEEE International Conference on Communications. ICC 2019, pp. 1–6 (2019). https://doi.org/10.1109/ICC.2019.8761349

  18. Wang, F., Wang, F., Liu, J., Shea, R., Sun, L.: Intelligent video caching at network edge: a multi-agent deep reinforcement learning approach. In: Proceedings of the IEEE Conference on Computer Communications. INFOCOM 2020, pp. 2499–2508 (2020). https://doi.org/10.1109/INFOCOM41043.2020.9155373

  19. Qu, Z., Ye, B., Tang, B., Guo, S., Lu, S., Zhuang, W.: Cooperative caching for multiple bitrate videos in small cell edges. IEEE Trans. Mob. Comput. 19(2), 288–299 (2020). https://doi.org/10.1109/TMC.2019.2893917

    Article  Google Scholar 

  20. Amazon Web Services, Inc.: Amazon EC2 pricing (2021). https://aws.amazon.com/cn/ec2/pricing/on-demand/

  21. Tran, T.X., Pandey, P., Hajisami, A., Pompili, D.: Collaborative multi-bitrate video caching and processing in Mobile-Edge Computing networks. In: Proceedings of the 13th Annual Conference on Wireless On-demand Network Systems and Services. WONS 2017, pp. 165–172 (2017). https://doi.org/10.1109/WONS.2017.7888772

  22. Lo, W.C., Fan, C.L., Lee, J., Huang, C.Y., Chen, K.T., Hsu, C.H.: 360\(^{\circ }\) video viewing dataset in head-mounted virtual reality. In: Proceedings of the 8th ACM on Multimedia Systems Conference. MMSys 2017, pp. 211–216 (2017). https://doi.org/10.1145/3083187.3083219

Download references

Author information

Authors and Affiliations

  1. School of Computer and Information, Hohai University, Nanjing, China

    Zengjie Tan & Yuanyuan Xu

Authors
  1. Zengjie Tan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuanyuan Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yuanyuan Xu .

Editor information

Editors and Affiliations

  1. Nanjing University of Aeronautics and Astronautics, Nanjing, China

    Zhe Liu

  2. Shanghai Jiao Tong University, Shanghai, China

    Fan Wu

  3. Missouri University of Science and Technology, Rolla, MO, USA

    Sajal K. Das

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tan, Z., Xu, Y. (2021). Deep Reinforcement Learning Based Collaborative Mobile Edge Caching for Omnidirectional Video Streaming. In: Liu, Z., Wu, F., Das, S.K. (eds) Wireless Algorithms, Systems, and Applications. WASA 2021. Lecture Notes in Computer Science(), vol 12937. Springer, Cham. https://doi.org/10.1007/978-3-030-85928-2_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85928-2_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85927-5

  • Online ISBN: 978-3-030-85928-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation