Abstract
As wireless networks continue to advance, virtual reality (VR) transmission over wireless connections is progressively transitioning from concept to practical application. Although this technology can significantly enhance the VR user experience, its development bottleneck lies in the computing capacity of devices and transmission latency. Considering the limited computational resources of VR devices for rendering tasks, multi-access edge computing (MEC) servers are introduced to provide powerful computing capabilities. To cope with transmission latency, reconfigurable intelligent surface (RIS) enhances links between base stations (BSs) and users. Based on these two technologies, we propose a RIS-assisted VR streaming model, where BSs are equipped with MEC servers to assist data rendering. Firstly, the user association, power control, and RIS phase shift optimization problems in the VR transmission system are jointly modeled and analyzed, establishing a long-term minimization of the interaction delay model. Secondly, by modeling the optimization problem as a Markov decision process (MDP), a joint optimization framework based on multi-agent deep reinforcement learning (MADRL) is proposed. In this framework, we have separately designed two dedicated algorithms for discrete and continuous variables. Furthermore, multiple agents can provide feedback based on user experience and cooperate with each other to improve the joint strategy. Finally, the performance and superiority of the proposed solution and algorithm are validated through simulation experiments in different application scenarios.
Similar content being viewed by others
Data availability
Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
References
Chaccour, C., Amer, R., Zhou, B., & Saad, W. (2019). On the reliability of wireless virtual reality at terahertz (THz) frequencies. In 2019 10th IFIP International Conference on New Technologies, Mobility and Security (NTMS) (pp. 1–5). IEEE
Chen, M., Semiari, O., Saad, W., Liu, X., & Yin, C. (2019). Federated echo state learning for minimizing breaks in presence in wireless virtual reality networks. IEEE Transactions on Wireless Communications, 19(1), 177–191.
Hu, F., Deng, Y., Saad, W., Bennis, M., & Aghvami, A. H. (2020). Cellular-connected wireless virtual reality: Requirements, challenges, and solutions. IEEE Communications Magazine, 58(5), 105–111.
Hsu, C.-H. (2020). Mec-assisted fov-aware and qoe-driven adaptive 360 video streaming for virtual reality. In 2020 16th international conference on mobility, sensing and networking (MSN) (pp. 291–298). IEEE
Dai, J., Zhang, Z., Mao, S., & Liu, D. (2019). A view synthesis-based 360\(^\circ\) VR caching system over MEC-enabled C-RAN. IEEE Transactions on Circuits and Systems for Video Technology, 30(10), 3843–3855.
Chen, J., Wang, S., Jia, J., Wang, Q., Yang, L., & Wang, X. (2023). Multi-objective oriented resource allocation in reconfigurable intelligent surface assisted HCNs. Ad Hoc Networks, 140, 103066. https://doi.org/10.1016/j.adhoc.2022.103066
Di Renzo, M., Zappone, A., Debbah, M., Alouini, M.-S., Yuen, C., De Rosny, J., & Tretyakov, S. (2020). Smart radio environments empowered by reconfigurable intelligent surfaces: How it works, state of research, and the road ahead. IEEE Journal on Selected Areas in Communications, 38(11), 2450–2525.
Zhou, H., Kong, L., Elsayed, M., Bavand, M., Gaigalas, R., Furr, S., & Erol-Kantarci, M. (2022) Hierarchical reinforcement learning for RIS-assisted energy-efficient RAN. In GLOBECOM 2022–2022 IEEE Global Communications Conference (pp. 3326–3331). https://doi.org/10.1109/GLOBECOM48099.2022.10001554
Wu, Q., & Zhang, R. (2019). Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming. IEEE Transactions on Wireless Communications, 18(11), 5394–5409.
Alsenwi, M., Abolhasan, M., & Lipman, J. (2022). Intelligent and reliable millimeter wave communications for RIS-aided vehicular networks. IEEE Transactions on Intelligent Transportation Systems, 23(11), 21582–21592. https://doi.org/10.1109/TITS.2022.3190101
Dong, S., Zhan, J., Hu, W., Mohajer, A., Bavaghar, M., & Mirzaei, A. (2023). Energy-efficient hierarchical resource allocation in uplink-downlink decoupled NOMA HetNets. IEEE Transactions on Network and Service Management, 20(3), 3380–3395. https://doi.org/10.1109/TNSM.2023.3239417
Du, J., Yu, F. R., Chu, X., Feng, J., & Lu, G. (2018). Computation offloading and resource allocation in vehicular networks based on dual-side cost minimization. IEEE Transactions on Vehicular Technology, 68(2), 1079–1092.
Chen, M., Challita, U., Saad, W., Yin, C., & Debbah, M. (2019). Artificial neural networks-based machine learning for wireless networks: A tutorial. IEEE Communications Surveys & Tutorials, 21(4), 3039–3071.
Cao, H., Zhu, Y., Zheng, G., & Yang, L. (2017). A novel optimal mapping algorithm with less computational complexity for virtual network embedding. IEEE Transactions on Network and Service Management, 15(1), 356–371.
Chakareski, J. (2020). Viewport-adaptive scalable multi-user virtual reality mobile-edge streaming. IEEE Transactions on Image Processing, 29, 6330–6342.
Du, J., Yu, F. R., Lu, G., Wang, J., Jiang, J., & Chu, X. (2020). MEC-assisted immersive VR video streaming over terahertz wireless networks: A deep reinforcement learning approach. IEEE Internet of Things Journal, 7(10), 9517–9529.
He, X., Xing, H., Chen, Y., & Nallanathan, A. (2018). Energy-efficient mobile-edge computation offloading for applications with shared data. In 2018 IEEE global communications conference (GLOBECOM) (pp. 1–6). IEEE
Liu, Y., Liu, J., Argyriou, A., & Ci, S. (2018). MEC-assisted panoramic VR video streaming over millimeter wave mobile networks. IEEE Transactions on Multimedia, 21(5), 1302–1316.
Sun, Y., Chen, Z., Tao, M., & Liu, H. (2018). Communication, computing and caching for mobile VR delivery: Modeling and trade-off. In 2018 IEEE international conference on communications (ICC) (pp. 1–6). IEEE
Zheng, C., Liu, S., Huang, Y., & Yang, L. (2020) MEC-enabled wireless VR video service: A learning-based mixed strategy for energy-latency tradeoff. In 2020 IEEE wireless communications and networking conference (WCNC) (pp. 1–6). IEEE
Ma, Y., Ota, K., & Dong, M. (2023). QoE Optimization for Virtual Reality Services in Multi-RIS-Assisted Terahertz Wireless Networks. IEEE Journal on Selected Areas in Communications. https://doi.org/10.1109/JSAC.2023.3345394
Chen, J., Xia, J., Jia, J., Yang, L., & Wang, X. (2023). Cooperative caching, rendering and beamforming for RIS-assisted wireless virtual reality networks. IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2023.3345354
Chaccour, C., Soorki, M.N., Saad, W., Bennis, M., & Popovski, P. (2020). Risk-based optimization of virtual reality over Terahertz reconfigurable intelligent surfaces. In ICC 2020–2020 IEEE international conference on communications (ICC) (pp. 1–6). https://doi.org/10.1109/ICC40277.2020.9149411
Liu, X., Deng, Y., Han, C., & Renzo, M. D. (2022). Learning-based prediction, rendering and transmission for interactive virtual reality in RIS-assisted terahertz networks. IEEE Journal on Selected Areas in Communications, 40(2), 710–724. https://doi.org/10.1109/JSAC.2021.3118405
Chen, J., Xing, H., Xiao, Z., Xu, L., & Tao, T. (2021). A DRL agent for jointly optimizing computation offloading and resource allocation in MEC. IEEE Internet of Things Journal, 8(24), 17508–17524. https://doi.org/10.1109/JIOT.2021.3081694
Xu, Z., Wang, Y., Tang, J., Wang, J., & Gursoy, M. C. (2017). A deep reinforcement learning based framework for power-efficient resource allocation in cloud RANs. In 2017 IEEE international conference on communications (ICC) (pp. 1–6). https://doi.org/10.1109/ICC.2017.7997286
Rahimi, A. M., Ziaeddini, A., & Gonglee, S. (2022). A novel approach to efficient resource allocation in load-balanced cellular networks using hierarchical DRL. Journal of Ambient Intelligence and Humanized Computing, 13(5), 2887–2901.
Lin, P., Song, Q., Yu, F. R., Wang, D., Jamalipour, A., & Guo, L. (2021). Wireless virtual reality in beyond 5G systems with the internet of intelligence. IEEE Wireless Communications, 28(2), 70–77. https://doi.org/10.1109/MWC.001.2000303
Yang, L., Jia, J., Chen, J., & Wang, X. (2021). Online reliability optimization for URLLC in HetNets: a DQN approach. Neural Computing and Applications, 33, 7271–7290.
Yang, L., Jia, J., Chen, J., & Wang, X. (2023) Joint power allocation and blocklength assignment for reliability optimization in CA-enabled HetNets. Peer-to-Peer Networking and Applications, 1–15
Yang, L., Jia, J., Chen, J., & Wang, X. (2022). Cooperative MARL for resource allocation in high mobility NGMA-enabled HetNets. In 2022 IEEE 96th vehicular technology conference (VTC2022-Fall) (pp. 1–5). https://doi.org/10.1109/VTC2022-Fall57202.2022.10012752
Wu, Q., & Zhang, R. (2019). Intelligent reflecting surface enhanced wireless network via joint active and passive beamforming. IEEE Transactions on Wireless Communications, 18(11), 5394–5409. https://doi.org/10.1109/TWC.2019.2936025
Liu, X., & Deng, Y. (2021). Learning-based prediction, rendering and association optimization for MEC-enabled wireless virtual reality (VR) networks. IEEE Transactions on Wireless Communications, 20(10), 6356–6370. https://doi.org/10.1109/TWC.2021.3073623
Wedel, S., Koppetz, M., Skowronek, J., & Raake, A. (2019). ViProVoQ: Towards a vocabulary for video quality assessment in the context of creative video production. In Proceedings of the 27th ACM international conference on multimedia (pp. 2387–2395)
Choi, L. U., Ivrlac, M. T., Steinbach, E., & Nossek, J. A. (2005). Sequence-level models for distortion-rate behaviour of compressed video. In IEEE international conference on image processing 2005 (vol. 2, p. 486). IEEE
Yang, H., Xiong, Z., Zhao, J., Niyato, D., Xiao, L., & Wu, Q. (2020). Deep reinforcement learning-based intelligent reflecting surface for secure wireless communications. IEEE Transactions on Wireless Communications, 20(1), 375–388.
Wei, Q., Lewis, F. L., Sun, Q., Yan, P., & Song, R. (2017). Discrete-time deterministic \(Q\) -learning: A novel convergence analysis. IEEE Transactions on Cybernetics, 47(5), 1224–1237. https://doi.org/10.1109/TCYB.2016.2542923
Yang, Z., Liu, Y., Chen, Y., & Al-Dhahir, N. (2021). Machine learning for user partitioning and phase shifters design in RIS-aided NOMA networks. IEEE Transactions on Communications, 69(11), 7414–7428. https://doi.org/10.1109/TCOMM.2021.3100866
Vinyals, O., Fortunato, M., & Jaitly, N. (2015). Pointer networks. Advances in Neural Information Processing Systems 28
Funding
This work was supported in part by the Major Research Plan of National Natural Science Foundation of China under Grant No. 92167103, in part by the National Natural Science Foundation of China under Grants No. 62172084, 62132004, 62032013, 61972079, in part by the Aeronautical Science Foundation of China under Grant No. 20230026050001, in part by the Young and Middle-Aged Leading Talents in Technological Innovation of Shenyang under Grant No. RC231173, in part by the Key Research and Development Program of LiaoNing under Grant No. 2023JH2/101300196, in part by the Fundamental Research Funds for the Central Universities under Grants No. N2324004-12, N2216009, N2216006, N2116004, and in part by the LiaoNing Revitalization Talents Program under Grant No. XLYC2007162.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no conflicts of interest to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Jia, J., Yang, L., Chen, J. et al. Online delay optimization for MEC and RIS-assisted wireless VR networks. Wireless Netw 30, 2939–2959 (2024). https://doi.org/10.1007/s11276-024-03706-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11276-024-03706-4