Skip to main content

Advertisement

Springer Nature Link
Log in

Deadline-Aware Cache Placement Scheme Using Fuzzy Reinforcement Learning in Device-to-Device Mobile Edge Networks

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Caching the most likely to be requested content at the mobile devices in a cooperative manner can facilitate direct content delivery without fetching content from the remote content server and thus alleviate the user-perceived latency, reduce the burden on backhaul and minimize the duplicated content transmissions. In addition to content popularity, it is also essential to consider the users’ dynamic behaviour for real-time applications, which can further improve the communication chances between user devices, leading to efficient content service time. The majority of previous studies consider stationary network topologies, in which all users remain stationary during data transmission, and the user can receive desired content from the corresponding base station. In this work, we study an essential issue: caching content by taking advantage of user mobility and the randomness of user interaction time. In a cooperative caching problem, we consider a realistic scenario with user devices moving at various velocities. We formulate the cache placement problem as maximization of saved delay with capacity and deadline constraints by considering the contact duration and inter-contact time among the user devices. We designed on-policy learning integrated fuzzy logic-based caching scheme to solve the high dimensionality of the proposed Integer linear programming problem. The proposed caching schemes improve the long-term reward and higher convergence rate than the Q-learning mechanism. Extensive simulation results demonstrate that the proposed cooperative caching mechanism significantly improves the performance in terms of reward, acceleration ratio, hit ratio and offloading ratio compared with existing mechanisms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Data Availability

The MovieLens 1M dataset was used to support this study and its application details are available in https://grouplens.org/datasets/movielens/1m/. The data set is cited at relevant places within the text as references.

Code Availability

The program of this paper is supported by custom code. It can be applied from the corresponding author on reasonable request.

References

  1. Wang X, Han Y, Wang C, Zhao Q, Chen X, Chen M (2019) In-edge ai: intelligentizing mobile edge computing, caching and communication by federated learning. IEEE Netw 33(5):156–165

    Article  Google Scholar 

  2. Inc CS (2019) Cisco visual networking index: global mobile data traffic forecast update, 2017—2022. White Paper

  3. Yao J, Han T, Ansari N (2019) On mobile edge caching. IEEE Commun Surv Tutor 21 (3):2525–2553

    Article  Google Scholar 

  4. Qiu L, Cao G (2019) Popularity-aware caching increases the capacity of wireless networks. IEEE Trans Mob Comput 19(1):173–187

    Article  Google Scholar 

  5. Pan Y, Pan C, Yang Z, Chen M, Wang J (2019) A caching strategy towards maximal d2d assisted offloading gain. IEEE Trans Mob Comput 19(11):2489–2504

    Article  Google Scholar 

  6. Prerna D, Tekchandani R, Kumar N (2020) Device-to-device content caching techniques in 5g: a taxonomy, solutions, and challenges. Comput Commun 153:48–84

    Article  Google Scholar 

  7. Yu S, Dab B, Movahedi Z, Langar R, Wang L (2019) A socially-aware hybrid computation offloading framework for multi-access edge computing. IEEE Trans Mob Comput 19(6):1247–1259

    Article  Google Scholar 

  8. Tran TX, Le DV, Yue G, Pompili D (2018) Cooperative hierarchical caching and request scheduling in a cloud radio access network. IEEE Trans Mob Comput 17(12):2729–2743

  9. Li L, Zhao G, Blum RS (2018) A survey of caching techniques in cellular networks: research issues and challenges in content placement and delivery strategies. IEEE Commun Surv Tutor 20(3):1710–1732

    Article  Google Scholar 

  10. Chen M, Saad W, Yin C, Debbah M (2017) Echo state networks for proactive caching in cloud-based radio access networks with mobile users. IEEE Trans Wirel Commun 16(6):3520–3535

    Article  Google Scholar 

  11. Zhu H, Cao Y, Wang W, Jiang T, Jin S (2018) Deep reinforcement learning for mobile edge caching: review, new features, and open issues. IEEE Netw 32(6):50–57

    Article  Google Scholar 

  12. Shanmugam K, Golrezaei N, Dimakis AG, Molisch AF, Caire G (2013) Femtocaching: wireless content delivery through distributed caching helpers. IEEE Trans Inf Theory 59(12):8402–8413

    Article  MathSciNet  Google Scholar 

  13. Li J, Liu M, Lu J, Shu F, Zhang Y, Bayat S, Jayakody DNK (2019) On social-aware content caching for d2d-enabled cellular networks with matching theory. IEEE Internet Things J 6(1):297–310

    Article  Google Scholar 

  14. Yang C, Stoleru R (2020) Ceo: cost-aware energy efficient mobile data offloading via opportunistic communication. In: 2020 International conference on computing, networking and communications (ICNC). IEEE, pp 548–554

  15. Wang Z, Shah-Mansouri H, Wong VW (2016) How to download more data from neighbors? a metric for d2d data offloading opportunity. IEEE Trans Mob Comput 16(6):1658–1675

    Article  Google Scholar 

  16. Wang R, Zhang J, Song S, Letaief KB (2017) Mobility-aware caching in d2d networks. IEEE Trans Wirel Commun 16(8):5001–5015

    Article  Google Scholar 

  17. Qiao J, He Y, Shen XS (2016) Proactive caching for mobile video streaming in millimeter wave 5g networks. IEEE Trans Wirel Commun 15(10):7187–7198

    Article  Google Scholar 

  18. Lu Z, Sun X, La Porta T (2016) Cooperative data offloading in opportunistic mobile networks. In: IEEE INFOCOM 2016-The 35th annual IEEE international conference on computer communications. IEEE, pp 1–9

  19. Zhou H, Wang H, Li X, Leung VC (2018) A survey on mobile data offloading technologies. IEEE Access 6:5101–5111

    Article  Google Scholar 

  20. Poularakis K, Iosifidis G, Tassiulas L (2014) Approximation algorithms for mobile data caching in small cell networks. IEEE Trans Commun 62(10):3665–3677

    Article  Google Scholar 

  21. Baştuğ E, Kountouris M, Bennis M, Debbah M (2016) On the delay of geographical caching methods in two-tiered heterogeneous networks. In: 2016 IEEE 17th International workshop on signal processing advances in wireless communications (SPAWC). IEEE, pp 1–5

  22. Somesula MK, Rout RR, Somayajulu D (2021) Contact duration-aware cooperative cache placement using genetic algorithm for mobile edge networks. Comput Netw 193:108062

    Article  Google Scholar 

  23. Bharath B, Nagananda KG, Gündüz D, Poor HV (2018) Caching with time-varying popularity profiles: a learning-theoretic perspective. IEEE Trans Commun 66(9):3837–3847

    Article  Google Scholar 

  24. Somesula MK, Rout RR, Somayajulu D (2021) Deadline-aware caching using echo state network integrated fuzzy logic for mobile edge networks. Wirel Netw, 1–21

  25. Wang X, Zhang Y, Leung VC, Guizani N, Jiang T (2018) D2d big data: content deliveries over wireless device-to-device sharing in large-scale mobile networks. IEEE Wirel Commun 25(1):32–38

    Article  Google Scholar 

  26. Ma X, Xu H, Gao H, Bian M (2021) Real-time multiple-workflow scheduling in cloud environments. IEEE Trans Netw Serv Manag 18(4):4002–4018

    Article  Google Scholar 

  27. Fu Y, Salaün L, Yang X, Wen W, Quek TQ (2021) Caching efficiency maximization for device-to-device communication networks: a recommend to cache approach. IEEE Transactions on Wireless Communications

  28. Wu D, Zhou L, Cai Y, Qian Y (2018) Collaborative caching and matching for d2d content sharing. IEEE Wireless Commun 25(3):43–49

    Article  Google Scholar 

  29. Hu J, Lennox B, Arvin F (2021) Collaborative coverage for a network of vacuum cleaner robots. In: Annual conference towards autonomous robotic systems. Springer, 112–115

  30. Hu J, Bhowmick P, Lanzon A (2021) Group coordinated control of networked mobile robots with applications to object transportation. IEEE Trans Veh Technol 70(8):8269–8274

    Article  Google Scholar 

  31. Liu JJ, Lam J, Kwok KW (2021) Further improvements on non-negative edge consensus of networked systems. IEEE Transactions on Cybernetics

  32. Liu Z, Song H, Pan D (2020) Distributed video content caching policy with deep learning approaches for d2d communication. IEEE Trans Veh Technol 69(12):15644–15655

    Article  Google Scholar 

  33. Zhao D, Wang H, Shao K, Zhu Y (2016) Deep reinforcement learning with experience replay based on sarsa. In: 2016 IEEE Symposium series on computational intelligence (SSCI). IEEE, pp 1–6

  34. Zhang S, Quan W, Li J, Shi W, Yang P, Shen X (2018) Air-ground integrated vehicular network slicing with content pushing and caching. IEEE J Selected Areas Commun 36(9):2114–2127

    Article  Google Scholar 

  35. Zhang W, Wu D, Yang W, Cai Y (2019) Caching on the move: a user interest-driven caching strategy for d2d content sharing. IEEE Trans Veh Technol 68(3):2958–2971

    Article  Google Scholar 

  36. Sun R, Wang Y, Lyu L, Cheng N, Zhang S, Yang T, Shen X (2020) Delay-oriented caching strategies in d2d mobile networks. IEEE Trans Veh Technol 69(8):8529–8541

    Article  Google Scholar 

  37. Ibrahim AM, Zewail AA, Yener A (2020) Device-to-device coded-caching with distinct cache sizes. IEEE Trans Commun 68(5):2748–2762

    Article  Google Scholar 

  38. Yin Y, Huang Q, Gao H, Xu Y (2020) Personalized apis recommendation with cognitive knowledge mining for industrial systems. IEEE Trans Industr Inform 17(9):6153–6161

    Article  Google Scholar 

  39. Huang Y, Xu H, Gao H, Ma X, Hussain W (2021) Ssur: an approach to optimizing virtual machine allocation strategy based on user requirements for cloud data center. IEEE Trans Green Commun Netw 5(2):670–681

    Article  Google Scholar 

  40. Gao H, Zhang Y, Miao H, Barroso RJD, Yang X (2021) Sdtioa: modeling the timed privacy requirements of iot service composition: a user interaction perspective for automatic transformation from bpel to timed automata. Mobile Networks and Applications, 1–26

  41. Gao H, Liu C, Yin Y, Xu Y, Li Y (2021) A hybrid approach to trust node assessment and management for vanets cooperative data communication: historical interaction perspective. IEEE Transactions on Intelligent Transportation Systems

  42. Poularakis K, Tassiulas L (2016) Code, cache and deliver on the move: a novel caching paradigm in hyper-dense small-cell networks. IEEE Trans Mob Comput 16(3):675–687

    Article  Google Scholar 

  43. Zhou H, Wu T, Zhang H, Wu J (2021) Incentive-driven deep reinforcement learning for content caching and d2d offloading. IEEE Journal on Selected Areas in Communications

  44. He Y, Liang C, Yu FR, Leung VC (2018) Integrated computing, caching, and communication for trust-based social networks: a big data drl approach. In: 2018 IEEE global communications conference (GLOBECOM). IEEE, pp 1–6

  45. Qiu X, Liu L, Chen W, Hong Z, Zheng Z (2019) Online deep reinforcement learning for computation offloading in blockchain-empowered mobile edge computing. IEEE Trans Veh Technol 68(8):8050–8062

    Article  Google Scholar 

  46. Zeng S, Ren Y, Wang Y, Zhao T, Qian Z (2019) Caching strategy based on deep q-learning in device-to-device scenario. In: 2019 12th International symposium on computational intelligence and design (ISCID), vol 1. IEEE, pp 175–179

  47. Li L, Xu Y, Yin J, Liang W, Li X, Chen W, Han Z (2019) Deep reinforcement learning approaches for content caching in cache-enabled d2d networks. IEEE Internet Things J 7(1):544–557

    Article  Google Scholar 

  48. Jiang W, Feng G, Qin S, Yum TSP, Cao G (2019) Multi-agent reinforcement learning for efficient content caching in mobile d2d networks. IEEE Trans Wirel Commun 18(3):1610–1622

    Article  Google Scholar 

  49. Somesula MK, Rout RR, Somayajulu D (2022) Cooperative cache update using multi-agent recurrent deep reinforcement learning for mobile edge networks. Comput Netw 209:108876. https://doi.org/10.1016/j.comnet.2022.108876, https://www.sciencedirect.com/science/article/pii/S1389128622000809

    Article  Google Scholar 

  50. Ahlehagh H, Dey S (2014) Video-aware scheduling and caching in the radio access network. IEEE/ACM Trans Netw (TON) 22(5):1444–1462

    Article  Google Scholar 

  51. Peng X, Shen JC, Zhang J, Letaief KB (2015) Backhaul-aware caching placement for wireless networks. arXiv:150900558

  52. Blaszczyszyn B, Giovanidis A (2015) Optimal geographic caching in cellular networks. In: 2015 IEEE International conference on communications (ICC). IEEE, pp 3358–3363

  53. Harper FM, Konstan JA (2015) The movielens datasets: history and context. ACM Trans Interact Intell Syst 5(4):19:1–19:19. https://doi.org/10.1145/2827872

    Google Scholar 

  54. Garg N, Sellathurai M, Bhatia V, Bharath B, Ratnarajah T (2019) Online content popularity prediction and learning in wireless edge caching. IEEE Trans Commun 68(2):1087–1100

    Article  Google Scholar 

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, BVRIT Hyderabad College of Engineering For Women, Hyderabad, 500090, Telangana, India

    Manoj Kumar Somesula

  2. Computer Science & Engineering, JNTUH College of Engineering Hyderabad, Hyderabad, 500085, India

    Anusha Kotte

  3. Department of Computer Science and Engineering, National Institute of Technology Calicut, Kerala, 673601, India

    Sudarshan Chakravarthy Annadanam

  4. Department of Computer Science and Engineering, National Institute of Technology Tiruchirappalli, Tamil Nadu, 620015, India

    Sai Krishna Mothku

Authors
  1. Manoj Kumar Somesula
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Anusha Kotte
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Sudarshan Chakravarthy Annadanam
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Sai Krishna Mothku
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Sai Krishna Mothku.

Ethics declarations

Conflict of Interests

The authors declare that they do not have any known competing interest.

Ethical statement

The work submitted by the authors is his own work and it is neither published nor considered for publication elsewhere.

Informed consent

Not applicable.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Somesula, M.K., Kotte, A., Annadanam, S.C. et al. Deadline-Aware Cache Placement Scheme Using Fuzzy Reinforcement Learning in Device-to-Device Mobile Edge Networks. Mobile Netw Appl 27, 2100–2117 (2022). https://doi.org/10.1007/s11036-022-02010-9

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-022-02010-9

Keywords