Abstract
Vehicular edge computing (VEC) supports real-time vehicular application services. However, accurate prediction and caching of popular content in roadside units (RSUs) while safeguarding user privacy is challenging. This study proposes a privacy-preserving cooperative hierarchical caching approach based on federated deep reinforcement learning (PCFR) for VEC. An asynchronous federated learning algorithm based on improved differential privacy is proposed, which considers the vehicle locations and movement directions and reasonably limits the local-update norm, improving the global model prediction accuracy while protecting user privacy. To address spatiotemporal variations in content popularity, a proposed attention-weighted asynchronous actor-critic collaborative caching algorithm extracts and weights key state features to optimize the collaborative cache content and its distribution location, enhancing the overall caching efficiency. In simulation, the PCFR scheme outperforms other caching schemes. With a 400-MB cache capacity, the PCFR scheme improves the cache hit rate by approximately 50.0% and reduces the content access delay by approximately 28.0%.



















Similar content being viewed by others
Data availability
The datasets analyzed during the current study were all derived from the following public domain resources [https://grouplens.org/datasets/movielens/; https://grouplens.org/datasets/hetrec-2011/].
References
Xiao H, Zhang X, Hu Z, Zheng M, Liang Y (2024) A collaborative cache allocation strategy for performance and link cost in mobile edge computing. J Supercomput 80(15):22885–22912. https://doi.org/10.1007/s11227-024-06310-7
Zhao J, Huang F, Liao L, Zhang Q (2024) Blockchain-based trust management model for vehicular ad hoc networks. IEEE Internet Things J 11(5):8118–8132. https://doi.org/10.1109/JIOT.2023.3318597
Tewari P, Tripathi S (2023) An energy efficient routing scheme in internet of things enabled wsn: neuro-fuzzy approach. J Supercomput 79(10):11134–11158. https://doi.org/10.1007/s11227-023-05091-9
Jeong J, Guo S, Gu Y, He T, Du DH (2012) Trajectory-based statistical forwarding for multihop infrastructure-to-vehicle data delivery. IEEE Trans Mob Comput 11(10):1523–1537. https://doi.org/10.1109/TMC.2011.189
Yin L, Cao G (2006) Supporting cooperative caching in ad hoc networks. IEEE Trans Mob Comput 5(1):77–89. https://doi.org/10.1109/TMC.2006.15
Singh P, Hazarika B, Singh K, Pan C, Huang WJ, Li CP (2024) Drl-based federated learning for efficient vehicular caching management. IEEE Internet Things J 11(21):34156–34171. https://doi.org/10.1109/JIOT.2024.3417265
Anh Nguyen DT, Cheng J, Nguyen DT, Nedić A (2023) Crowdcache: A decentralized game-theoretic framework for mobile edge content sharing. In: 2023 21st International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), pp 350–357, https://doi.org/10.23919/WiOpt58741.2023.10349816
Huang Y, Song X, Ye F, Yang Y, Li X (2020) Fair and efficient caching algorithms and strategies for peer data sharing in pervasive edge computing environments. IEEE Trans Mob Comput 19(4):852–864. https://doi.org/10.1109/TMC.2019.2902090
Liu J, Zhu B, Wang F, Jin Y, Zhang W, Xu Z, Cui S (2023) Cav3: Cache-assisted viewport adaptive volumetric video streaming. In: 2023 IEEE Conference Virtual Reality and 3D User Interfaces (VR), pp 173–183, https://doi.org/10.1109/VR55154.2023.00033
Li Y, Ma H, Wang L, Mao S, Wang G (2022) Optimized content caching and user association for edge computing in densely deployed heterogeneous networks. IEEE Trans Mob Comput 21(6):2130–2142. https://doi.org/10.1109/TMC.2020.3033563
Zeng F, Zhang K, Wu L, Wu J (2023) Efficient caching in vehicular edge computing based on edge-cloud collaboration. IEEE Trans Veh Technol 72(2):2468–2481. https://doi.org/10.1109/TVT.2022.3213130
Yu Z, Hu J, Min G, Wang Z, Miao W, Li S (2022) Privacy-preserving federated deep learning for cooperative hierarchical caching in fog computing. IEEE Internet Things J 9(22):22246–22255. https://doi.org/10.1109/JIOT.2021.3081480
Zhao M, Nakhai MR (2024) A unified federated deep q learning caching scheme for scalable collaborative edge networks. IEEE Transactions on Mobile Computing pp 1–12. https://doi.org/10.1109/TMC.2024.3382824
Tian A, Feng B, Zhou H, Huang Y, Sood K, Yu S, Zhang H (2023) Efficient federated drl-based cooperative caching for mobile edge networks. IEEE Trans Netw Serv Manage 20(1):246–260. https://doi.org/10.1109/TNSM.2022.3198074
Wang S, Wu Q, Fan Q, Fan P, Wang J (2023) Mobility-aware asynchronous federated learning for edge-assisted vehicular networks. In: ICC 2023 - IEEE International Conference on Communications, pp 3621–3626, https://doi.org/10.1109/ICC45041.2023.10278823
Zhao J, Huang C, Wang W, Xie R, Dong R, Matwin S (2023) Local differentially private federated learning with homomorphic encryption. J Supercomput 79(17):19365–19395. https://doi.org/10.1007/s11227-023-05378-x
Yang X, Huang W, Ye M (2023) Dynamic personalized federated learning with adaptive differential privacy. Adv Neural Inf Process Syst 36:72181–72192
Du R, Wu J, Gao Y (2024) Dual-q network deep reinforcement learning-based computation offloading method for industrial internet of things. J Supercomput 80(17):25590–25615. https://doi.org/10.1007/s11227-024-06425-x
Cao T, Zhang Z, Wang X, Xiao H, Xu C (2024) Ptcc: A privacy-preserving and trajectory clustering-based approach for cooperative caching optimization in vehicular networks. IEEE Trans Sustain Comput 9(4):615–630. https://doi.org/10.1109/TSUSC.2024.3350386
Liu M, Li D, Wu H, Lyu F, Shen X (2022) Real-time search-driven caching for sensing data in vehicular networks. IEEE Internet Things J 9(14):12219–12230. https://doi.org/10.1109/JIOT.2021.3134964
Wu H, Jin J, Ma H, Xing L (2024) Federation-based deep reinforcement learning cooperative cache in vehicular edge networks. IEEE Internet Things J 11(2):2550–2560. https://doi.org/10.1109/JIOT.2023.3292374
Liu Y, Mao B (2023) On a novel content edge caching approach based on multi-agent federated reinforcement learning in internet of vehicles. In: 2023 32nd Wireless and Optical Communications Conference (WOCC), pp 1–5, https://doi.org/10.1109/WOCC58016.2023.10139417
Somesula MK, Rout RR, Somayajulu D (2023) Greedy cooperative cache placement for mobile edge networks with user preferences prediction and adaptive clustering. Ad Hoc Netw 140:103051. https://doi.org/10.1016/j.adhoc.2022.103051
Yang S, Fan S, Deng G, Tian H (2019) Local content cloud based cooperative caching placement for edge caching. In: 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pp 1–6, https://doi.org/10.1109/PIMRC.2019.8904138
Li Q, Wang X, Wang D (2019) Optimal d2d cooperative caching system in sdn based wireless network. In: 2019 IEEE 30th Annual International Symposium on Personal, Indoor and Mobile Radio Communications (PIMRC), pp 1–7, https://doi.org/10.1109/PIMRC.2019.8904149
Jin Y, Liu J, Wang F, Cui S (2023) Ebublio: Edge-assisted multiuser \(360^\circ\) video streaming. IEEE Internet Things J 10(17):15408–15419. https://doi.org/10.1109/JIOT.2023.3263885
Yu Z, Hu J, Min G, Zhao Z, Miao W, Hossain MS (2021) Mobility-aware proactive edge caching for connected vehicles using federated learning. IEEE Trans Intell Transp Syst 22(8):5341–5351. https://doi.org/10.1109/TITS.2020.3017474
Li D, Zhang H, Li T, Ding H, Yuan D (2023) Community detection and attention-weighted federated learning based proactive edge caching for d2d-assisted wireless networks. IEEE Trans Wireless Commun 22(11):7287–7303. https://doi.org/10.1109/TWC.2023.3249756
Wang W, Zhao Y, Wu Q, Fan Q, Zhang C, Li Z (2022) Asynchronous federated learning based mobility-aware caching in vehicular edge computing. In: 2022 14th International Conference on Wireless Communications and Signal Processing (WCSP), pp 1–5, https://doi.org/10.1109/WCSP55476.2022.10039430
Zhou X, Liang W, Kawai A, Fueda K, She J, Wang KIK (2024) Adaptive segmentation enhanced asynchronous federated learning for sustainable intelligent transportation systems. IEEE Trans Intell Transp Syst 25(7):6658–6666. https://doi.org/10.1109/TITS.2024.3362058
Cao XK, Wang CD, Lai JH, Huang Q, Chen CLP (2023) Multiparty secure broad learning system for privacy preserving. IEEE Trans Cybernet 53(10):6636–6648. https://doi.org/10.1109/TCYB.2023.3235496
Nguyen CH, Saputra YM, Hoang DT, Nguyen DN, Nguyen VD, Xiao Y, Dutkiewicz E (2024) Encrypted data caching and learning framework for robust federated learning-based mobile edge computing. IEEE/ACM Trans Netw 32(3):2705–2720. https://doi.org/10.1109/TNET.2024.3365815
Jiang X, Liu B, Qin J, Zhang Y, Qian J (2022) Fedncf: Federated neural collaborative filtering for privacy-preserving recommender system. In: 2022 International Joint Conference on Neural Networks (IJCNN), pp 1–8, https://doi.org/10.1109/IJCNN55064.2022.9892909
Cheng A, Wang P, Zhang XS, Cheng J (2022) Differentially private federated learning with local regularization and sparsification. In: 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp 10112–10121, https://doi.org/10.1109/CVPR52688.2022.00988
Gao X, Sun Y, Chen H, Xu X, Cui S (2024) Joint computing, pushing, and caching optimization for mobile-edge computing networks via soft actor-critic learning. IEEE Internet Things J 11(6):9269–9281. https://doi.org/10.1109/JIOT.2023.3323433
Qiao D, Guo S, Liu D, Long S, Zhou P, Li Z (2022) Adaptive federated deep reinforcement learning for proactive content caching in edge computing. IEEE Trans Parallel Distrib Syst 33(12):4767–4782. https://doi.org/10.1109/TPDS.2022.3201983
Li C, Zhang Y, Luo Y (2023) A federated learning-based edge caching approach for mobile edge computing-enabled intelligent connected vehicles. IEEE Trans Intell Transp Syst 24(3):3360–3369. https://doi.org/10.1109/TITS.2022.3224395
Yang W, Liu Z (2024) Efficient vehicular edge computing: a novel approach with asynchronous federated and deep reinforcement learning for content caching in vec. IEEE Access 12:13196–13212. https://doi.org/10.1109/ACCESS.2024.3355462
Singh P, Hazarika B, Singh K, Pan C, Huang WJ, Li CP (2024) Drl-based federated learning for efficient vehicular caching management. IEEE Internet Things J 11(21):34156–34171. https://doi.org/10.1109/JIOT.2024.3417265
3GPP A (2016) Study on lte-based v2x services. 3rd Generation Partnership Project (3GPP), TR 36885
Chen J, Wu H, Yang P, Lyu F, Shen X (2020) Cooperative edge caching with location-based and popular contents for vehicular networks. IEEE Trans Veh Technol 69(9):10291–10305. https://doi.org/10.1109/TVT.2020.3004720
Lee HS, Lee JW (2021) Adaptive transmission scheduling in wireless networks for asynchronous federated learning. IEEE J Sel Areas Commun 39(12):3673–3687. https://doi.org/10.1109/JSAC.2021.3118353
Wu Q, Wang W, Fan P, Fan Q, Zhu H, Letaief KB (2024) Cooperative edge caching based on elastic federated and multi-agent deep reinforcement learning in next-generation networks. IEEE Trans Netw Serv Manage 21(4):4179–4196. https://doi.org/10.1109/TNSM.2024.3403842
Harper FM, Konstan JA (2015) The movielens datasets: history and context. Acm Trans Interact Intell Syst (tiis) 5(4):1–19
Wu Q, Zhao Y, Fan Q, Fan P, Wang J, Zhang C (2023) Mobility-aware cooperative caching in vehicular edge computing based on asynchronous federated and deep reinforcement learning. IEEE J Sel Topics Signal Process 17(1):66–81. https://doi.org/10.1109/JSTSP.2022.3221271
Cantador I, Brusilovsky P, Kuflik T (2011) 2nd workshop on information heterogeneity and fusion in recommender systems (hetrec 2011). In: Proceedings of the 5th ACM conference on Recommender systems. ACM, New York, NY, USA, RecSys 2011
Acknowledgements
This work was supported by Songshan Laboratory (Project No. 232102210154), the Pre-research Project SongShan Laboratory (Project No. YYJC022022001), and the Major Science and Technology Projects in Henan Province (Project No. 241110210200).
Author information
Authors and Affiliations
Contributions
Yangxi Mu worked in investigation, conceptualization, methodology, simulation, validation, writing—original draft, visualization, and writing—review & editing. Mengyang He helped in conceptualization, methodology, validation, writing—original draft, and writing—review & editing. Bing Hao helped in methodology, validation, and writing—review & editing.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Consent to Participate
The authors declare that they agree to participate.
Consent for Publication
The authors declare that they agree to publish.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Proof of theorem 1
Proof of theorem 1
Theorem 1
(The Gaussian mechanism satisfies \((\epsilon , \delta )\)-differential privacy) Given privacy parameters \(\epsilon > 0\) and \(\delta \in (0,1)\), the Gaussian mechanism \(\mathcal {M}(D) = f(D) + \mathcal {N}(0, \sigma ^2 I)\) satisfies \((\epsilon , \delta )\)-DP when the noise standard deviation \(\sigma\) satisfies
Here, \(\Delta f\) is the \(L_2\)-sensitivity of function f(x), representing the maximum output difference between any two adjacent datasets D and \(D'\); i.e., \(\Delta f = \max _{D, D'} \Vert f(D) - f(D') \Vert _2\).
Proof of Theorem 1
Assuming that the Gaussian mechanism is \(\mathcal {M}(D) = f(D) + \mathcal {N}(0, \sigma ^2 I)\), for any output value z, its probability density function is:
Similarly, the probability density function of \(D'\) is:
By comparing their output probabilities, we have:
Simplifying this expression gives:
Considering the properties of the Gaussian distribution, when the values of f(D) or \(f(D')\) approach z, the maximum value is:
AS the maximum difference between D and \(D'\) is \(\Vert f(D) - f(D')\Vert _2 \le \Delta f\), it follows that:
To further meet the relaxed condition of \((\epsilon , \delta )\)-differential privacy, the privacy loss function \(\mathcal {L}(o; \mathcal {M}, D, D')\) is introduced to analyze the privacy leakage. Here,
This function measures the difference in the algorithm output distribution on adjacent datasets. When this function is combined with the Rényi differential privacy method, solution of the high-order moments of the privacy loss function derives the upper bound for the noise standard deviation required to satisfy the \((\epsilon , \delta )\)-differential privacy condition:
This formula incorporates the function sensitivity \(\Delta f\), privacy budget \(\epsilon\), and relaxation parameter \(\delta\), ensuring more stringent privacy protection within the permissible privacy disclosure range. \(\hfill\square\)
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Mu, Y., Hao, B. & He, M. Privacy-preserving cooperative hierarchical caching approach based on federated deep reinforcement learning for vehicular edge computing. J Supercomput 81, 532 (2025). https://doi.org/10.1007/s11227-025-07058-4
Accepted:
Published:
DOI: https://doi.org/10.1007/s11227-025-07058-4