Skip to main content

Advertisement

Log in

A GRL-aided federated graph reinforcement learning approach for enhanced file caching in mobile edge computing

  • Regular Paper
  • Published:
Computing Aims and scope Submit manuscript

Abstract

The exponential growth of data and user density in mobile networks has led to increased latency in delivering content, particularly in mobile edge computing (MEC). Edge caching strategies, which store frequently accessed data closer to users, are crucial to minimize latency. However, traditional caching methods struggle to adapt to the dynamic environment of MEC, where user mobility and data popularity are highly variable. This paper addresses the problem of optimizing caching strategies to reduce content transmission delay in MEC while preserving user privacy. We propose a federated graph reinforcement learning (CFGRL) approach that integrates graph neural networks with federated learning and deep reinforcement learning (DRL). This model predicts popular content and updates caching strategies dynamically, improving latency without sharing user data. Simulations on real-world datasets show that CFGRL achieves a 32% reduction in content transmission delay and an 11.9% increase in cache hit ratio compared to existing methods like CFDRL and Thompson sampling. The CFGRL model demonstrates superior scalability and efficiency, making it well-suited for real-time applications in highly dynamic mobile environments. These results suggest that the proposed approach can significantly enhance the performance of MEC systems, making it a vital solution for modern mobile networks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4
Algorithm 2
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data availability

No datasets were generated or analysed during the current study.

References

  1. Abbas N, Zhang Y, Taherkordi A, Skeie T (2017) Mobile edge computing: a survey. IEEE Internet Things J 5(1):450–465. https://doi.org/10.1109/JIOT.2017.2750180

    Article  MATH  Google Scholar 

  2. Yi S, Li C, Li Q (2015) A survey of fog computing: concepts, applications and issues. In: Proceeding of workshop mobile big data, pp 37–42. https://doi.org/10.1145/2757384.2757397

  3. Jararweh Y, Doulat A, Darabseh A, Alsmirat M, Al-Ayyoub M, Benkhelifa E (2016) SDMEC: software defined system for mobile edge computing. In: Proceeding of IEEE international conference on cloud engineering workshop (IC2EW), pp 88–93. https://doi.org/10.1109/IC2EW.2016.45

  4. Zhang S, He P, Suto K, Yang P, Zhao L, Shen X (2018) Cooperative edge caching in user-centric clustered mobile networks. IEEE Trans Mobile Comput 17(8):1791–1805. https://doi.org/10.1109/TMC.2017.2780834

    Article  Google Scholar 

  5. Wu Q, Zhao Y, Fan Q (2022) Time-dependent performance modeling for platooning communications at intersection. IEEE Internet Things J 9(19):18500–18513. https://doi.org/10.1109/JIOT.2022.3161028

    Article  MATH  Google Scholar 

  6. Wu Q, Wan Z, Fan Q, Fan P, Wang J (2022) Velocity-adaptive ac- cess scheme for MEC-assisted platooning networks: access fairness via data freshness. IEEE Internet Things J 9(6):4229–4244. https://doi.org/10.1109/JIOT.2021.3103325

    Article  MATH  Google Scholar 

  7. Dai Y, Xu D, Maharjan S, Qiao G, Zhang Y (2019) Artificial intelligence empowered edge computing and caching for internet of vehicles. IEEE Wireless Commun Mag 26(3):12–18. https://doi.org/10.1109/MWC.2019.1800411

    Article  MATH  Google Scholar 

  8. Xu W, Yang Z, Ng D, Levorato M, Eldar YC, Debbah M (2022) Edge learning for B5G networks with distributed signal processing: Semantic communication, edge computing, and wireless sensing. IEEE J Sel Top Signal Process. https://doi.org/10.1109/JSTSP.2023.3239189

    Article  Google Scholar 

  9. Javed MA, Zeadally S (2021) AI-Empowered content caching in vehicular edge computing: opportunities and challenges. IEEE Netw 35(3):109–115. https://doi.org/10.1109/MNET.011.2000561

    Article  MATH  Google Scholar 

  10. Narayanan A, Verma S, Ramadan E, Babaie P, Zhang ZL (2018) DeepCache: a deep learning based framework for content caching. In: Proceedings of workshop network meets AI ML NetAI, pp 48–53. https://doi.org/10.1145/3229543.3229555

  11. Chen M, Yang Z, Saad W, Yin C, Poor HV, Cui S (2021) A joint learning and communications framework for federated learning over wireless networks. IEEE Trans Wireless Commun 20(1):269–283. https://doi.org/10.1109/JIOT.2020.2986803

    Article  MATH  Google Scholar 

  12. Wang X, Wang C, Li X, Leung VCM, Taleb T (2020) Federated deep reinforcement learning for Internet of Things with decentralized cooperative edge caching. IEEE Internet Things J 7(10):9441–9455. https://doi.org/10.1109/TWC.2020.3024629

    Article  MATH  Google Scholar 

  13. Cui L et al (2022) CREAT: Blockchain-assisted compression algorithm of federated learning for content caching in edge computing. IEEE Internet Things J 9(16):14151–14161. https://doi.org/10.1109/JIOT.2020.3014370

    Article  MATH  Google Scholar 

  14. Cheng R, Sun Y, Liu Y, Xia L, Feng D, Imran M (2022) Blockchain-empowered federated learning approach for an intelligent and reliable D2D caching scheme. IEEE Internet Things J 9(11):7879–7890. https://doi.org/10.1109/JIOT.2021.3103107

    Article  MATH  Google Scholar 

  15. Manzoor S, Mian AN, Zoha A, Imran MA (2022) Federated learning empowered mobility-aware proactive content offloading framework for fog radio access networks. Future Gener Comput Syst 133:307–319. https://doi.org/10.1016/j.future.2022.03.025

    Article  Google Scholar 

  16. Wang Y, Zheng Ke, Ye W, Tang Y (2023) Popularity-aware caching for vehicle clusters with federated deep reinforcement learning. IEEE Commun Lett 27(6):1644–1648. https://doi.org/10.1109/LCOMM.2023.3267141

    Article  MATH  Google Scholar 

  17. Yang F, Yang C, Huang J, Alfarraj O, Tolba A, Yu K, Guizani M (2024) Mutual interference-aware throughput enhancement in massive IoT: a graph reinforcement learning framework. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2024.3411653

    Article  Google Scholar 

  18. Jiang B, Wang L, Cheng J, Tang J, Luo B (2021) Gpens: graph data learning with graph propagation-embedding networks. IEEE Trans Neural Netw Learning Syst. https://doi.org/10.1109/TNNLS.2021.3120100

    Article  MATH  Google Scholar 

  19. Dai Y, Xu D, Zhang K, Maharjan S, Zhang Y (2020) Deep reinforcement learning and permissioned blockchain for content caching in vehicular edge computing and networks. IEEE Trans Veh Technol 69(4):4312–4324. https://doi.org/10.1109/TVT.2020.2973705

    Article  MATH  Google Scholar 

  20. Qiao G, Leng S, Maharjan S, Zhang Y, Ansari N (2020) Deep reinforcement learning for cooperative content caching in vehicular edge computing and networks. IEEE Internet Things J 7(1):247–257. https://doi.org/10.1109/JIOT.2019.2945640

    Article  MATH  Google Scholar 

  21. Somesula MK, Mothku SK, Annadanam SC (2023) Cooperative service placement and request routing in mobile edge networks for latency-sensitive applications. IEEE Syst J 17(3):4050–4061. https://doi.org/10.1109/JSYST.2023.3260028

    Article  Google Scholar 

  22. Zhang W, Zhang G, Mao S (2023) Deep reinforcement learning based joint caching and resources allocation for cooperative MEC. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2023.3333826

    Article  MATH  Google Scholar 

  23. Lekharu A, Samanta A, Sur A, Patra M (2024) Content-aware caching at the mobile edge network using federated learning. IEEE Trans Emerg Top Comput Intell. https://doi.org/10.1109/TETCI.2024.3402166

    Article  Google Scholar 

  24. Liu M, Li D, Wu H, Lyu F, Shen XS (2022) Real-time search-driven caching for sensing data in vehicular networks. IEEE Internet Things J 9(14):12219–12230. https://doi.org/10.1109/JIOT.2021.3134964

    Article  MATH  Google Scholar 

  25. Dai Y, Zhang Y (2022) Adaptive digital twin for vehicular edge computing and networks. J Commun Inf Netw 7(1):48–59. https://doi.org/10.23919/JCIN.2022.9745481

    Article  MATH  Google Scholar 

  26. Luo Q, Li C, Luan TH, Shi W (2020) Collaborative data scheduling for vehicular edge computing via deep reinforcement learning. IEEE Internet Things J 7(10):9637–9650. https://doi.org/10.1109/JIOT.2020.2983660

    Article  MATH  Google Scholar 

  27. Zhang K, Cao J, Liu H, Maharjan S, Zhang Y (2020) Deep reinforcement learning for social-aware edge computing and caching in urban informatics. IEEE Trans Ind Inform 16(8):5467–5477. https://doi.org/10.1109/TII.2019.2953189

    Article  MATH  Google Scholar 

  28. Qiao D, Guo S, Liu D, Long S, Zhou P, Li Z (2022) Adaptive federated deep reinforcement learning for proactive content caching in edge computing. IEEE Trans Parallel Distrib Syst 33(12):4767–4782. https://doi.org/10.1109/TPDS.2022.3201983

    Article  MATH  Google Scholar 

  29. Zhao L, Ran Y, Wang H, Wang J, Luo J (2021) Towards cooperative caching for vehicular networks with multi-level federated reinforcement learning. In: Proceedings IEEE international conference on communications, pp 1–6. https://doi.org/10.1109/ICC42927.2021.9500714 .

  30. Zhu J, Huang X, Shao Z (2020) Learning-aided content placement in caching-enabled fog computing systems using thompson sampling. In: ICASSP 2020–2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 5060–5064. https://doi.org/10.1109/ICASSP40776.2020.9053162

  31. McMahan B, Moore E, Ramage D, Hampson S, y Arcas BA (2017) Communication-efficient learning of deep networks from decentralized data. In: Artificial intelligence and statistics 2017. PMLR, pp 1273–1282. https://doi.org/10.48550/arXiv.1602.05629

  32. Wang YALI (2022) Collaborative caching in edge computing via federated learning and deep reinforcement learning. Wireless Commun Mobile Comput 2:1–15. https://doi.org/10.3390/electronics11233968

    Article  MATH  Google Scholar 

  33. Hu Z, Fang C, Wang Z, Tseng SM, Dong M (2023) Many-objective optimization based-content popularity prediction for cache-assisted cloud-edge-end collaborative IoT networks. IEEE Internet Things J. https://doi.org/10.1109/JIOT.2023.3290793

    Article  MATH  Google Scholar 

  34. Wu Q, Zhao Y, Fan Q, Fan P, Wang J, Zhang C (2022) Mobility-aware cooperative caching in vehicular edge computing based on asynchronous federated and deep reinforcement learning. IEEE J Sel Top Signal Processing. https://doi.org/10.48550/arXiv.2208.01219

    Article  MATH  Google Scholar 

  35. Mirzaei R, Arani MG, Esmaeili L (2023) A review on the edge caching mechanisms in the mobile edge computing: a social-aware perspective. J Internet Things. https://doi.org/10.1016/j.iot.2023.100690

    Article  MATH  Google Scholar 

  36. Chunlin L, Zhang Y, Luo Y (2023) DQN-enabled content caching and quantum ant colony-based computation offloading in MEC. Appl Soft Comput 133:109900. https://doi.org/10.1016/j.asoc.2022.109900

    Article  MATH  Google Scholar 

  37. Yang Y, Lou K, Wang En, Liu W, Shang J, Song X, Li D, Jie Wu (2023) Multi-agent reinforcement learning based file caching strategy in mobile edge computing. IEEE/ACM Trans Netwo 31(6):3159–3174. https://doi.org/10.1109/TNET.2023.3278032

    Article  MATH  Google Scholar 

  38. Hou J, Xia H, Lu H, Nayak A (2021) A GNN-based approach to optimize cache hit ratio in NDN networks. In: Proceedings of the 2021 IEEE global communications conference (GLOBECOM), Madrid, Spain, pp 1–6. https://doi.org/10.21203/rs.3.rs-1713271/v1

  39. Hou J, Tao T, Lu H, Nayak A (2023) Intelligent caching with graph neural network-based deep reinforcement learning on SDN-based ICN. Future Internet 15(8):251. https://doi.org/10.3390/fi15080251

    Article  Google Scholar 

  40. Hou J, Lu H, Nayak A (202) GNN-GM: a proactive caching scheme for named data networking. In: Proceedings of the 2022 IEEE international conference on communications workshops (ICC workshops). https://doi.org/10.21203/rs.3.rs-1713271/v1

  41. Di Y, Liu Y (2023) MFPCDR: a meta-learning-based model for federated personalized cross-domain recommendation[J]. Appl Sci 13(7):4407. https://doi.org/10.3390/app13074407

    Article  MATH  Google Scholar 

  42. Di Y, Shi H, Wang X, Ma R, Liu Y (2024) Federated recommender system based on diffusion augmentation and guided denoising. ACM Trans Inf Syst. https://doi.org/10.1145/3688570

    Article  MATH  Google Scholar 

  43. Di Y, Shi H, Ma R, Gao H, Liu Y, Wang W (2024) FedRL: a reinforcement learning federated recommender system for efficient communication using reinforcement selector and hypernet generator. ACM Trans Recomm Syst. https://doi.org/10.1145/3682076

    Article  Google Scholar 

  44. Yu Z, Hu J, Min G, Zhao Z, Miao W, Hossain MS (2021) Mobility aware proactive edge caching for connected vehicles using federated learning. IEEE Trans Intell Transp Syst 22(8):5341–5351. https://doi.org/10.1109/TITS.2020.3017474

    Article  MATH  Google Scholar 

  45. Liu C et al (2022) Learning-based predictive beamforming for integrated sensing and communication in vehicular networks. IEEE J Sel Areas Commun 40(8):2317–2334. https://doi.org/10.1109/JSAC.2022.3180803

    Article  MATH  Google Scholar 

  46. Cheng X, Duan D, Gao S, Yang L (2022) Integrated sensing and communications (ISAC) for vehicular communication networks (VCN). IEEE Internet Things J. https://doi.org/10.1109/JIOT.2022.3191386

    Article  MATH  Google Scholar 

  47. Zhu H, Wang J (2009) Chunk-based resource allocation in OFDMA systems—Part I: chunk allocation. IEEE Trans Commun 57(9):2734–2744. https://doi.org/10.1109/TCOMM.2009.09.080067

    Article  MATH  Google Scholar 

  48. Zhu H, Wang J (2012) Chunk-based resource allocation in OFDMA systems—Part II: Joint chunk, power and bit allocation. IEEE Trans Commun 60(2):499–509. https://doi.org/10.1109/TCOMM.2011.112811.110036

    Article  MATH  Google Scholar 

  49. Mir ZH, Dreyer N, Kürner T, Filali F (2024) Investigation on cellular LTE C-V2X network serving vehicular data traffic in realistic urban scenarios. Future Gener Comput Syst 161(2024):66–80. https://doi.org/10.1016/j.future.2024.07.002

    Article  Google Scholar 

  50. Liang L, Ye H, Li GY (2019) Spectrum sharing in vehicular networks based on multi-agent reinforcement learning. IEEE J Sel Areas Commun 37(10):2282–2292. https://doi.org/10.1109/JSAC.2019.2933962

    Article  MATH  Google Scholar 

  51. Liu C, Liu X, Ng DWK, Yuan J (2022) Deep residual learning for channel estimation in intelligent reflecting surface-assisted multi-user communications. IEEE Trans Wireless Commun 21(2):898–912. https://doi.org/10.1109/TWC.2021.3100148

    Article  MATH  Google Scholar 

  52. Neumann D, Wiese T, Utschick W (2018) Learning the MMSE channel estimator. IEEE Trans Signal Process 66(11):2905–2917. https://doi.org/10.1109/TSP.2018.2799164

    Article  MathSciNet  MATH  Google Scholar 

  53. Chen J, Wu H, Yang P, Lyu F, Shen X (2020) Cooperative edge caching with location-based and popular contents for vehicular networks. IEEE Trans Veh Technol 69(9):10291–10305. https://doi.org/10.1109/TVT.2020.3004720

    Article  MATH  Google Scholar 

  54. Ng A (2011) Sparse autoencoder. CS294A Lect Notes 72:1–19

    MATH  Google Scholar 

  55. Chen Y, Ning Y, Slawski M, Rangwala H (2020) Asynchronous online federated learning for edge devices with non-IID data. In: Proceedings of IEEE International Conference on Big Data, pp 15–24. https://doi.org/10.1109/BigData50022.2020.9378161

  56. Xie C, Koyejo S, Gupta I (2019) Asynchronous federated optimization. arXiv preprint arXiv:1903.03934. https://doi.org/10.48550/arXiv.1903.03934

  57. Lee H-S, Lee J-W (2021) Adaptive transmission scheduling in wireless networks for asynchronous federated learning. IEEE J Sel Areas Commun 39(12):3673–3687. https://doi.org/10.1109/JSAC.2021.3118353

    Article  MATH  Google Scholar 

  58. Huang J, Yang C, Zhang S, Yang F, Alfarraj O, Frascolla V, Mumtaz S, Yu K (2024) Reinforcement learning based resource management for 6G-enabled mIoT with hypergraph interference model. In: IEEE transactions on communications. https://doi.org/10.1109/TCOMM.2024.3372892

  59. Zeng X, Zhou T, Bao Z, Zhao H, Chen L, Wang X, Wang F (2022) Feature-contrastive graph federated learning: responsible AI in graph information analysis. IEEE Trans Comput Soc Syst 10(6):2938–2948. https://doi.org/10.1109/TCSS.2022.3230987

    Article  MATH  Google Scholar 

  60. Li C, Ke Z, Liu Q et al (2023) Energy–latency tradeoffs edge server selection and DQN-based resource allocation schemes in MEC. Wireless Netw 29:3637–3663. https://doi.org/10.1007/s11276-023-03426-1

    Article  MATH  Google Scholar 

  61. Umemoto K (2022) ML-1M++: MovieLens-compatible additional preferences for more robust offline evaluation of sequential recommenders. In: Proceedings of the 31st ACM international conference on information & knowledge management, pp 4540–4544. https://doi.org/10.1145/3511808.355764

Download references

Funding

It’s not funded by any agency or organization either technically or financially.

Author information

Authors and Affiliations

Authors

Contributions

All authors contributed equally to the work. Abhinav Khanna—concept, methodology, data collection, implementation, visibility, testing & validation, article draft. Gandikota Anjali—concept, methodology, data collection, implementation, visibility, testing & validation, article draft. Nilesh Kumar Verma—concept, methodology, validation, article revision. K. Jairam Naik—overall supervision, article review & correction.

Corresponding author

Correspondence to K. Jairam Naik.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Ethical approval

All authors read before submission and approved the final manuscript for submission and agreed to the terms and conditions of this journal.

Consent for publication

All authors gave explicit consent to publish this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Khanna, A., Anjali, G., Verma, N.K. et al. A GRL-aided federated graph reinforcement learning approach for enhanced file caching in mobile edge computing. Computing 107, 40 (2025). https://doi.org/10.1007/s00607-024-01396-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s00607-024-01396-6

Keywords

Mathematics Subject Classification

Navigation