Skip to main content
SpringerLink
Log in

A Q-learning-based approach for virtual network embedding in data center

  • Deep Learning & Neural Computing for Intelligent Sensing and Control
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Virtual network embedding (VNE) refers to allocating reasonable substrate network resources for virtual network (VN) requests that include computing resources and network resources, so as to obtain optimal income from leasing virtual resources. Such a way of providing virtual resources is the key technology of cloud computing and can greatly save the operating cost of enterprises and provide flexibility of application deployment. However, the existing VNE algorithms are mostly oriented to traditional stochastic network topologies. Due to the high connectivity and server density of data centers and the complexity of the user’s resource requirements, the traditional VNE algorithms suffer from low resource utilization rate and revenues in the VNE on the data centers. Different from the existing algorithms which are often based on heuristic algorithms, this paper proposes a VNE algorithm for data center topology based on the Q-learning algorithm which is a typical reinforcement learning method. The algorithm an agent for each VN designs a reward function related to the effect of virtual link embedding, which is used to update the Q-matrix through unsupervised learning process. Then, the agent can find the optimal embedding strategy based on the Q-table from each learning. Simulation results demonstrate that the proposed algorithm can improve the resource utilization ratio and obtain a better revenue/cost ratio of the substrate network compared with the traditional heuristic algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Lu FF, Xie XH, Guo DK et al (2017) Modular network structure for building mega-modular data center. J Softw 28(8):2196–2213

    Google Scholar 

  2. Yang J, Wang GY, Liu Q et al (2018) Retrospect and prospect of research of normal cloud model. Chin J Comput 3:724–744

    Google Scholar 

  3. Wang C, Liu G, Peng S et al (2017) Virtual network embedding with pre-transformation and incentive convergence mechanism. Concurr Comput Pract Exp 29(14):1–21

    Article  Google Scholar 

  4. Zheng HK, Li JJ, Gong YJ, et al (2017) Link mapping-oriented ant colony system for virtual network embedding. In: IEEE congress on evolutionary computation, pp 1223–1230

  5. Truong-Huu T, Gurusamy M (2017) Markov chain based algorithm for virtual network embedding in optical data centers. In: IEEE international conference on high performance computing and communications, pp 899–906

  6. Fischer A, Botero JF, Beck MT et al (2013) Virtual network embedding: a survey. IEEE Commun Surv Tutor 15(4):1888–1906

    Article  Google Scholar 

  7. Lu J, Turner J (2006) Efficient mapping of virtual networks onto a shared substrate, WUCSE-2006-35. Department of Computer Science and Engineering, Washington University, Washington

    Google Scholar 

  8. Pathak I, Vidyarthi DP (2015) An optimal virtual network mapping model based on dynamic threshold. Wirel Pers Commun 3:1–21

    Google Scholar 

  9. Zhang P, Yao H, Liu Y (2016) Virtual network embedding based on the degree and clustering coefficient information. IEEE Access 4:8572–8580

    Article  Google Scholar 

  10. Ogino N, Kitahara T, Arakawa S, Murata M (2017) Virtual net- work embedding with multiple priority classes sharing substrate resources. Comput Netw 112:52–66

    Article  Google Scholar 

  11. Liao J, Feng M, Qing S, Li T, Wang J (2016) Live: learning and inference for virtual network embedding. J Netw Syst Manag 24(2):227–256

    Article  Google Scholar 

  12. Gong S, Chen J, Zhao S, Zhu Q (2016) Virtual network em-bedding with multi-attribute node ranking based on TOPSIS. In: KSII transactions on internet and information systems, vol 10

  13. Zhu F, Wang H (2015) A modified ACO algorithm for virtual network embedding based on graph decomposition. Comput Commun 80:1–15

    Article  MathSciNet  Google Scholar 

  14. Hu Y, Zhuang L, Lan J et al (2016) Energy aware virtual network embedding using particle swarm optimization algorithm based on adaptive cooperative coevolution. J Electron Inf Technol 38(10):2667–2673

    Google Scholar 

  15. Zhang P, Yao H, Fang C, Liu Y (2016) Multi-objective enhanced particle swarm optimization in virtual network embedding. EURASIP J Wirel Commun Netw 2016(1):167

    Article  Google Scholar 

  16. Zhangbao Z, Cheng X, Su S et al (2013) A unified enhanced particle swarm optimization-based virtual network embedding algorithm. Int J Commun Syst 26(8):1054–1073

    Article  Google Scholar 

  17. Zhang Q, Lin M, Yang LT, et al (2017) Energy-efficient scheduling for real-time systems based on deep Q-learning model. In: IEEE transactions on sustainable computing, vol 99, p 1

  18. Cherkassky BV, Goldberg AV, Radzik T (1996) Shortest path algorithms: theory and experimental evaluation. Math Program 73(2):129–174

    Article  MathSciNet  Google Scholar 

  19. Calheiros RN, Ranjan R, Beloglazov A et al (2011) CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms. Softw Pract Exp 41(1):23–50

    Article  Google Scholar 

  20. Vahdat A, Al-Fares M, Loukissas A (2013) Scalable commodity data center network architecture: US, US8483096

  21. Yang AM, Yang XL, Chang JC, Bai B, Kong FB, Ran QB (2018) Research on a fusion scheme of cellular network and wireless sensor for cyber physical social systems. IEEE Access 6(1):18786–18794

    Article  Google Scholar 

  22. Lu DJ, Huang XX, Zhang GJ, Zheng XW, Liu H (2018) Trusted device-to-device based heterogeneous cellular networks: a new framework for connectivity optimization. IEEE Trans Veh Technol 67(11):11219–11233

    Article  Google Scholar 

  23. Zhou X, Liang X, Du X, Zhao J (2018) Structure based user identification across social networks. IEEE Trans Knowl Data Eng 30(6):1178–1191

    Article  Google Scholar 

  24. Koley E, Verma K, Ghosh S (2017) A modular neuro-wavelet based non-unit protection scheme for zone identification and fault location in six-phase transmission line. Neural Comput Appl 28:1369. https://doi.org/10.1007/s00521-016-2566-3

    Article  Google Scholar 

  25. Li X, He JM, Hou BJ, Zhang PY (2018) Exploring the innovation modes and evolution of the cloud-based service using the activity theory on the basis of big data. Clust Comput 21(1):907–922

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundation of China under Grant No. 61702089, the Basic scientific research operating fund of central universities under Grant No. N182304021 and the Scientific research plan for institutions of higher learning of Hebei province under Grant No. ZD2019306.

Author information

Authors and Affiliations

  1. Computing Center, Northeastern University at Qinhuangdao, Qinhuangdao, 066004, Hebei, China

    Ying Yuan

  2. Department of Computer and Communication Engineering, Northeastern University at Qinhuangdao, Qinhuangdao, 066004, Hebei, China

    Zejie Tian, Cong Wang, Fanghui Zheng & Yanxia Lv

Authors
  1. Ying Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zejie Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fanghui Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yanxia Lv
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cong Wang.

Ethics declarations

Conflict of interest

There are no conflicts of interests of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yuan, Y., Tian, Z., Wang, C. et al. A Q-learning-based approach for virtual network embedding in data center. Neural Comput & Applic 32, 1995–2004 (2020). https://doi.org/10.1007/s00521-019-04376-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04376-6

Keywords

Search

Navigation