Abstract
Vehicles request road condition information, traffic information, and various audio-visual entertainment frequently. Repeat Download will burden the core network and seriously affect the user experience. Edge caching is a promising technology that can effectively alleviate the pressure of repeatedly downloading content from the cloud. There are many existing edge cache scheduling methods, but they all have limitations. his paper proposes an edge cache scheduling method based on the multi-head attention mechanism federal reinforcement learning (FRLMA). Firstly, the problem is modeled as a Markov decision model. The local models are trained through a deep reinforcement learning method. Finally, the federated reinforcement learning framework of edge Cooperative Cache is established. In particular, the multi-head attention mechanism is introduced to weigh the contribution of the local model to the global model from multiple angles. Simulation results show that the FRLMA method has better convergence and is superior to the most current popular methods in terms of hit rate and average delay.
Supported by the Natural Science Foundation of Anhui Province (2108085MF202).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ren, K., Wang, Q., Wang, C., Qin, Z., Lin, X.: The security of autonomous driving: threats, defenses, and future directions. Proc. IEEE 108(2), 357–372 (2020)
Yang, W., He, S., Fan, X., Xu, C., Sun, X.H.: On cost-driven collaborative data caching: a new model approach. IEEE Trans. Parallel Distrib. Syst. 30, 662–676 (2018)
Li, K., Kumar, S., Lloyd, W.A., Huang, Q., Tang, L.: RIPQ: advanced photo caching on flash for facebook. USENIX Association (2015)
Jia, W.: A survey of web caching schemes for the internet. ACM SIGCOMM Comput. Commun. Rev. 29(5), 36–46 (2000)
Rossi, D., Rossini, G., Paristech, T., Paris, F.: Caching performance of content centric networks under multi-path routing (and more) (2011)
Zhao, J., Sun, X., Li, Q., Ma, X.: Edge caching and computation management for real-time internet of vehicles: an online and distributed approach. IEEE Trans. Intell. Transp. Syst. 22(4), 2183–2197 (2021)
Pang, H., Liu, J., Fan, X., Sun, L.: Toward smart and cooperative edge caching for 5G networks: a deep learning based approach. In: 2018 IEEE/ACM 26th International Symposium on Quality of Service (IWQoS) (2018)
Hasselt, H.V., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. Computer Science (2015)
Li, R., Zhao, Y., Wang, C., Wang, X., Taleb, T.: Edge caching replacement optimization for d2d wireless networks via weighted distributed DQN. In: 2020 IEEE Wireless Communications and Networking Conference (WCNC) (2020)
Bonawitz, K., et al.: Towards federated learning at scale: system design (2019)
Tang, H., Ding, Z.: Mixed mode transmission and resource allocation for d2d communication. IEEE Trans. Wirel. Commun. 15(1), 1–1 (2016)
Vaswani, A., et al.: Attention is all you need. arXiv (2017)
Hefeeda, M., Saleh, O.: Traffic modeling and proportional partial caching for peer-to-peer systems. IEEE/ACM Trans. Netw. 16(6), 1447–1460 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, X., Wei, Z., lyu, Z., Yuan, X., Xu, J., Zhang, Z. (2022). Federated Reinforcement Learning Based on Multi-head Attention Mechanism for Vehicle Edge Caching. In: Wang, L., Segal, M., Chen, J., Qiu, T. (eds) Wireless Algorithms, Systems, and Applications. WASA 2022. Lecture Notes in Computer Science, vol 13473. Springer, Cham. https://doi.org/10.1007/978-3-031-19211-1_54
Download citation
DOI: https://doi.org/10.1007/978-3-031-19211-1_54
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19210-4
Online ISBN: 978-3-031-19211-1
eBook Packages: Computer ScienceComputer Science (R0)