Skip to main content
Springer Nature Link
Log in

A Reinforcement Learning Based Data Storage and Traffic Management in Information-Centric Data Center Networks

  • Published:
Mobile Networks and Applications Aims and scope Submit manuscript

Abstract

Data Center Networks (DCN), a core infrastructure of cloud computing, place heavy demands on efficient storage and management of massive data. The data storage scheme, which decides how to assign data to nodes for storage, has a significant impact on the performance of the data center. However, most of the existing solutions focus on where to store the data (i.e., the selection of storage node) but have not considered how to store them (i.e., the traffic management such as routing and transmission rate adjustment). By leveraging the Information-Centric Networks (ICN) architecture, this paper tackles the data storage and traffic management issue in Information-Centric Data Center Networks (ICDCN) based on Reinforcement Learning (RL) method, since RL has been developed as a promising solution to address dynamic network issues. We present a global optimization of joint traffic management and data storage and then solve it by the distributed multi-agent Q-learning. In ICDCN, the data is routed based on the data’s name, which achieves better routing scalability by decoupling the data and its physical location. Compared with IP’s stateless forwarding plane, the stateful forwarding information maintained at every node supports adaptively routing and hop-by-hop traffic control by using the Q-learning method. We evaluate our proposal on an NS-3-based simulator, and the results show that the proposed scheme can effectively reduce transmission time and increase throughput while achieving load-balanced among servers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Xia W, Zhao P, Wen Y, Xie H (2017) A survey on data center networking (DCN): infrastructure and operations. IEEE Communications Surveys Tutorials 19:640–656. https://doi.org/10.1109/COMST.2016.2626784

    Article  Google Scholar 

  2. Ko, B.J., Pappas, V., Raghavendra, R., Song, Y., Dilmaghani, R.B., Lee, K., Verma, D.: An information-centric architecture for data center networks. In: Proceedings of the second edition of the ICN workshop on Information-centric networking - ICN ‘12. p. 79. ACM Press, Helsinki, Finland (2012). https://doi.org/10.1145/2342488.2342506

  3. Pianese, F.: Information Centric Networks for Parallel Processing in the Datacenter. In: 2013 IEEE 33rd International Conference on Distributed Computing Systems Workshops. pp. 208–213. IEEE, Philadelphia, PA, USA (2013). https://doi.org/10.1109/ICDCSW.2013.67

  4. Zhu M, Li D, Wang F, Li A, Ramakrishnan KK, Liu Y, Wu J, Zhu N, Liu X (2016) CCDN: content-centric data center networks. IEEE/ACM Trans Networking 24:3537–3550. https://doi.org/10.1109/TNET.2016.2530739

    Article  Google Scholar 

  5. Zhou, Y., Ren, Y., Zhou, X., Li, Z., Fan, P.: A Congestion Control Mechanism for Data Center Networks Based on Named Data Networking. In: Proceedings of the 13th International Conference on Future Internet Technologies - CFI 2018. pp. 1–6. ACM Press, Seoul, Republic of Korea (2018). https://doi.org/10.1145/3226052.3226055

  6. Xie R, Wen Y, Jia X, Xie H (2015) Supporting seamless virtual machine migrati on via named data networking in cloud data center. IEEE Trans. Parallel Distrib. Syst. 26:3485–3497. https://doi.org/10.1109/TPDS.2014.2377119

    Article  Google Scholar 

  7. Zhu M, Li D, Liu Y, Wu J (2014) CDRDN: content driven routing in datacenter network. In: 2014 23rd international conference on computer communication and networks (ICCCN). Pp. 1–8. IEEE. China. https://doi.org/10.1109/ICCCN.2014.6911754

  8. Mansour, D., Tschudin, C.: Towards a Monitoring Protocol Over Information-Centric Networks. In: Proceedings of the 2016 conference on 3rd ACM Conference on Information-Centric Networking - ACM-ICN ‘16. pp. 60–64. ACM Press, Kyoto, Japan (2016). https://doi.org/10.1145/2984356.2984378

  9. Costa, P., Donnelly, A., O’Shea, G., Rowstron, A.: CamCubeOS: a key-based network stack for 3D torus cluster topologies. In: Proceedings of the 22nd international symposium on High-performance parallel and distributed computing - HPDC ‘13. p. 73. ACM Press, New York, New York, USA (2013). https://doi.org/10.1145/2493123.2462917

  10. Ghemawat, S., Gobioff, H., Leung, S.-T.: The Google File System. In: Proceedings of the 19th ACM Symposium on Operating Systems Principles. pp. 20–43. , Bolton Landing, NY (2003)

  11. Lakshman A, Malik P (2010) Cassandra: a decentralized structured storage system. SIGOPS Oper Syst Rev 44:35–40. https://doi.org/10.1145/1773912.1773922

    Article  Google Scholar 

  12. Shvachko, K., Kuang, H., Radia, S., Chansler, R.: The Hadoop Distributed File System. In: 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST). pp. 1–10 (2010). https://doi.org/10.1109/MSST.2010.5496972

  13. Renuga, K., Tan, S.S., Zhu, Y.Q., Low, T.C., Wang, Y.H.: Balanced and Efficient Data Placement and Replication Strategy for Distributed Backup Storage Systems. In: 2009 International Conference on Computational Science and Engineering. pp. 87–94 (2009). https://doi.org/10.1109/CSE.2009.27

  14. Zaman S, Grosu D (2011) A distributed algorithm for the replica placement problem. IEEE Transactions on Parallel and Distributed Systems 22:1455–1468. https://doi.org/10.1109/TPDS.2011.27

    Article  Google Scholar 

  15. Rajalakshmi, A., Vijayakumar, D., Srinivasagan, K.G.: An improved dynamic data replica selection and placement in cloud. In: 2014 International Conference on Recent Trends in Information Technology. pp. 1–6 (2014). https://doi.org/10.1109/ICRTIT.2014.6996180

  16. Vilaça, R., Oliveira, R., Pereira, J.: A correlation-aware data placement strategy for key-value stores. In: IFIP International Conference on Distributed Applications and Interoperable Systems. pp. 214–227. Springer (2011)

  17. Meroufel B, Belalem G (2012) Dynamic replication based on availability and popularity in the presence of failures. Journal of Information Processing Systems 8:263–278

    Article  Google Scholar 

  18. Paiva J, Ruivo P, Romano P, Rodrigues L (2015) A Uto p lacer: scalable self-tuning data placement in distributed key-value stores. ACM Transactions on Autonomous and Adaptive Systems (TAAS) 9(19)

  19. Wu J-J, Lin Y-F, Liu P (2008) Optimal replica placement in hierarchical data grids with locality assurance. Journal of Parallel and Distributed Computing 68:1517–1538

    Article  Google Scholar 

  20. Gao, C., Wang, H., Zhai, L., Gao, Y., Yi, S.: An energy-aware ant colony algorithm for network-aware virtual machine placement in cloud computing. In: 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS). pp. 669–676. IEEE (2016)

  21. Weil, S.A., Brandt, S.A., Miller, E.L., Maltzahn, C.: CRUSH: Controlled, Scalable, Decentralized Placement of Replicated Data. In: SC ‘06: Proceedings of the 2006 ACM/IEEE Conference on Supercomputing. pp. 31–31 (2006). https://doi.org/10.1109/SC.2006.19

  22. Qiao Lian, Wei Chen, Zheng Zhang: On the Impact of Replica Placement to the Reliability of Distributed Brick Storage Systems. In: 25th IEEE International Conference on Distributed Computing Systems (ICDCS’05). pp. 187–196 (2005). https://doi.org/10.1109/ICDCS.2005.56

  23. Liu, C., Chu, X., Liu, H., Leung, Y.-W.: ESet: Placing Data Towards Efficient Recovery for Large-Scale Erasure-Coded Storage Systems. In: 2016 25th International Conference on Computer Communication and Networks (ICCCN). pp. 1–9. IEEE, Waikoloa, HI (2016). https://doi.org/10.1109/ICCCN.2016.7568521

  24. Liu C, Wang Q, Chu X, Leung Y-W, Liu H (2020) ESetStore: an erasure-coded storage system with fast data recovery. IEEE Trans. Parallel Distrib. Syst. 31:2001–2016. https://doi.org/10.1109/TPDS.2020.2983411

    Article  Google Scholar 

  25. Abebe M, Daudjee K, Glasbergen B, Tian Y (2018) EC-store: bridging the gap between storage and latency in distributed erasure coded systems. In: 2018 IEEE 38th international conference on distributed computing systems (ICDCS). Pp. 255–266. IEEE. Vienna. https://doi.org/10.1109/ICDCS.2018.00034

  26. Zhirong, S., Patrick P. C., L., Jiwu, S., Wenzhong, G.: Cross-Rack-Aware Single Failure Recovery for Clustered File Systems. IEEE Transactions on Dependable and Secure Computing. 17, 248–261 (2020)

  27. Xia Q, Xu Z, Liang W, Yu S, Guo S, Zomaya AY (2019) Efficient data placement and replication for QoS-aware approximate query evaluation of big data analytics. IEEE Trans. Parallel Distrib. Syst. 30:2677–2691. https://doi.org/10.1109/TPDS.2019.2921337

  28. Liu K, Peng J, Wang J, Liu W, Huang Z, Pan J (2020) Scalable and adaptive data replica placement for geo-distributed cloud storages. IEEE Trans Parallel Distrib Syst 31:1575–1587. https://doi.org/10.1109/TPDS.2020.2968321

  29. Weihong, Y., Yang, Q., Zhaozheng, Y.: A Reinforcement Learning based Placement Strategy in Datacenter Networks. In: 15th EAI International Conference on Heterogeneous Networking for Quality, Reliability, Security and Robustness. pp. 1–16

  30. Mastorakis S, Afanasyev A, Zhang L (2017) On the evolution of ndnSIM. ACM SIGCOMM Computer Communication Review 47:15

    Article  Google Scholar 

  31. Jacobson, V., Smetters, D.K., Thornton, J.D., Plass, M.F., Briggs, N.H., Braynard, R.L.: Networking named content. In: Proceedings of the 5th international conference on Emerging networking experiments and technologies. pp. 1–12 (2009)

  32. Zhang L, Claffy K, Crowley P, Papadopoulos C, Wang L, Zhang B (2014) Named data networking. ACM SIGCOMM Computer Communication Review 44:66–73

    Article  Google Scholar 

  33. Al-Fares, M., Loukissas, A., Vahdat, A.: A Scalable, Commodity Data Center Network Architecture. In: Proceedings of the ACM SIGCOMM 2008 Conference on Data Communication. pp. 63–74. ACM, New York, NY, USA (2008). https://doi.org/10.1145/1402958.1402967

  34. Galindo-Serrano, A., Giupponi, L.: Distributed Q-Learning for Interference Control in OFDMA-Based Femtocell Networks. In: 2010 IEEE 71st Vehicular Technology Conference. pp. 1–5 (2010). https://doi.org/10.1109/VETECS.2010.5493950

  35. Saad, H., Mohamed, A., ElBatt, T.: A cooperative Q-learning approach for distributed resource allocation in multi-user femtocell networks. In: 2014 IEEE Wireless Communications and Networking Conference (WCNC). pp. 1490–1495. IEEE, Istanbul, Turkey (2014). https://doi.org/10.1109/WCNC.2014.6952410

  36. Yi C, Afanasyev A, Moiseenko I, Wang L, Zhang B, Zhang L (2013) A case for stateful forwarding plane. Comput Commun 36:779–791. https://doi.org/10.1016/j.comcom.2013.01.005

    Article  Google Scholar 

  37. Schneider, K., Yi, C., Zhang, B., Zhang, L.: A Practical Congestion Control Scheme for Named Data Networking. In: Proceedings of the 2016 conference on 3rd ACM Conference on Information-Centric Networking - ACM-ICN ‘16. pp. 21–30. ACM Press, Kyoto, Japan (2016). https://doi.org/10.1145/2984356.2984369

  38. Hoque, A.K.M.M., Amin, S.O., Alyyan, A., Zhang, B., Zhang, L., Wang, L.: NISR: named-data link state routing protocol. In: Proceedings of the 3rd ACM SIGCOMM workshop on Information-centric networking - ICN ‘13. p. 15. ACM Press, Hong Kong, China (2013). https://doi.org/10.1145/2491224.2491231

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Harbin Institute of Technology (Shenzhen), Shenzhen, China

    Weihong Yang, Yang Qin & ZhaoZheng Yang

Authors
  1. Weihong Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yang Qin
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. ZhaoZheng Yang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yang Qin.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, W., Qin, Y. & Yang, Z. A Reinforcement Learning Based Data Storage and Traffic Management in Information-Centric Data Center Networks. Mobile Netw Appl 27, 266–275 (2022). https://doi.org/10.1007/s11036-020-01629-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11036-020-01629-w

Keywords