Skip to main content
SpringerLink
Log in

Virtual network function deployment algorithm based on graph convolution deep reinforcement learning

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

In Network Function virtualization (NFV), network functions are virtualized as Virtual Network functions (VNFs). A network service consists of a set of VNFs. One of the major challenges in implementing this paradigm is allocating optimal resources to VNFs. Most existing work assumes that services are represented as service functional chains (SFC), which are chains. However, for more complex and diversified network services, a more appropriate representation is Virtual Network Function Forwarding Graph (VNF-FG), namely directed acyclic Graph (DAGs). Previous works failed to take advantage of this special graph structure, which makes them unsuitable for complex and diverse network service scenarios. Aiming at the problem of virtual network function Forwarding Graph mapping (VNF-FGE) represented by DAG, this paper proposes a virtual network function (VNF) deployment algorithm based on graph convolution network (GCN) and deep reinforcement learning (DRL), called GDRL-VNFP. First, we describe the VNF-FGE problem as an integer linear programming (ILP) problem and the virtual network service request (NSR) as a DAG. Secondly, in order to overcome the challenges brought about by the different sizes and dynamic arrival of the DAG representation of NSR, an efficient algorithm based on GCN and DRL is proposed, which can generate deployment solutions in real time under the premise of meeting the quality of service and minimize the total cost of deployment. We use GCN to extract features from the physical network topology and VNF-FG represented by DAG, and in addition, and construct a sequence-to-sequence model based on the attention mechanism for the output mapping scheme. Finally, we trained the model using a policy-based gradient-based reinforcement learning algorithm that could quickly find a near-optimal solution for the problem instance. Simulation results show that our GDRL-VNFP outperforms state-of-the-art solutions in terms of deployment cost, service request reception rate and runtime.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

  1. Network Functions Virtualisation (NFV); Architectural Framework, document ETSI GS NFV 002 (V1.2.1), 2014.

  2. Alam I et al (2020) A survey of network virtualization techniques for Internet of Things using SDN and NFV. ACM Comput Surv 53(2):1–35

    Article  Google Scholar 

  3. Mijumbi R et al (2015) Network function virtualization: state-of-the-art and research challenges. IEEE Commun Surv Tutorials 18(1):236–262

    Article  Google Scholar 

  4. Herrera JG, Botero JF (2016) Resource allocation in NFV: a comprehensive survey. IEEE Trans Netw Service Manag 13(3):518–532

    Article  Google Scholar 

  5. Laghrissi A, Taleb T (2019) "A survey on the placement of virtual resources and virtual network functions. IEEE Commun Surv Tuts 21(2):1409–1434

    Article  Google Scholar 

  6. Yu M, Yi Y, Rexford J, Chiang M (2008) “Rethinking virtual network embedding: substrate support for path splitting and migration.” ACM SIGCOMM Comput Commun Rev 38(2):17–29

    Article  Google Scholar 

  7. Wang M, Cheng B, Chen J (2020) Joint availability guarantee andresource optimization of virtual network function placement in datacenter networks. IEEE Trans Netw Service Manag 17(2):821–834

    Article  Google Scholar 

  8. Jin P, Fei X, Zhang Q, Liu F, Li B (2020) “Latency-aware VNF chaindeployment with efficient resource reuse at network edge.” Proc IEEE Int Conf Comput Commun (INFOCOM) 20:267–276

    Google Scholar 

  9. Zheng Z et al. (2019) “Octans: Optimal placement of service function chainsin many-core systems,” in Procceeding IEEE Internationsl Conference Computer Communication (INFOCOM), pp. 307–315.

  10. Soualah O, Fajjari I, Aitsaadi N, an Mellouk A (2014) “A reliable virtual network embedding algorithm based on game theory within cloud’s backbone,” in Procceedings IEEE International Conference Communication (ICC), pp. 2975–2981.

  11. Gong L, Wen Y, Zhu Z, and Lee T (2014) “Toward profit-seeking virtual network embedding algorithm via global resource capacity,” in Procceedings IEEE INFOCOM-IEEE Conference Computer Communication, pp. 1–9.

  12. Ouedraogo CA, Medjiah S, Chassot C, Drira K, Aguilar J (2021) A cost-effective approach for end-to-end QoS management in NFV enabled IoT platforms. IEEE Internet Things J 8(5):3885–3903

    Article  Google Scholar 

  13. Thiruvasagam PK, Kotagi VJ, Murthy SR (2020) “A reliability-aware, delay guaranteed, and resource efficient placement of service functionchains in softwarized 5G networks. IEEE Trans Cloud Comput, Earlyaccess 25:51. https://doi.org/10.1109/TCC.2020.3020269

    Article  Google Scholar 

  14. Zheng D, Peng C, Guler E, Luo G, Tian L, and Cao X (2019) “Hybridservice chain deployment in networks with unique function,” in Proccceedings IEEE International Conference Communication (ICC), pp. 1–6.

  15. Cheng X et al (2011) Virtual network embedding through topology-aware node ranking. ACM SIGCOMM Comput Commun Rev 41(2):38–47

    Article  Google Scholar 

  16. Page L, Brin S, Motwani R, Winograd T (1999) “The pagerank citation ranking: bringing order to the Web. Stanford InfoLab

    Google Scholar 

  17. Zhang Q, Liu F, and Zeng C (2019) “Adaptive interference-aware VNF placement for service-customized 5G network slices,” in Procceeding IEEE International Conference Computer Communication (INFOCOM), pp. 2449–2457.

  18. Houidi O, Soualah O, Louati W, Zeghlache D (2020) Dynamic VNF forwarding graph extension algorithms. IEEE Trans Netw Service Manag 17(3):1389–1402

    Article  Google Scholar 

  19. Deudon M, Cournut P, Lacoste A, Adulyasak Y, and Rousseau L-M (2018) "Learning heuristics for the TSP by policy gradient," in Procceedings International Conference Integration Constraint Programming, Artificial Intelligence, Operations Research Amsterdam, The Netherlands: Springer, 2018, pp. 170–181.

  20. Mirhoseini A et al. (2017) "Device placement optimization with reinforcement learning," in Procceedings 34th International Conference Machine Learning, vol. 70, 2017, pp. 2430–2439.

  21. Nazari M, Oroojlooy A, Snyder L, and Takác M (2018) "Reinforcement learning for solving the vehicle routing problem," in Procceedigs Advances Neural Information Processing System, pp. 9839–9849.

  22. Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press

    MATH  Google Scholar 

  23. Xiao Y et al (2019) NFVdeep: adaptive online service function chain deployment with deep reinforcement learning. Proc Int Symp Qual Serv (IWQoS) 21:1–21

    Google Scholar 

  24. Solozabal R, Ceberio J, Sanchoyerto A, Zabala L, Blanco B, Liberal F (2020) Virtual network function placement optimization with deepreinforcement learning. IEEE J Sel Areas Commun 38(2):292–303

    Article  Google Scholar 

  25. Fu X, Yu FR, Wang J, Qi Q, Liao J (2020) Dynamic service functionchain embedding for NFV-enabled IoT: a deep reinforcement learningapproach. IEEE Trans Wireless Commun 19(1):507–519

    Article  Google Scholar 

  26. Kipf TN and Welling M (2016) “Semi-supervised classification with graphconvolutional networks,” arXiv preprint arXiv:1609.02907

  27. Ilya S, Oriol V, Quoc VL (2014) Sequence to sequence learning with neuralnetworks. Adv Neural Inf Process Syst 20(3104–3112):2014

    Google Scholar 

  28. Oriol V, Samy B, and Manjunath K (2016) Order matters: Sequence to sequence forsets.

  29. Mnih V et al. (2016) “Asynchronous methods for deep reinforcement learning,” In Procceedings International Conference Machine Learning, 2016, pp. 1928–1937.

  30. Gong L, Wen Y, Zhu Z, and Lee T (2014) “Toward profit-seeking virtualnetwork embedding algorithm via global resource capacity,” in Procceedings IEEE INFOCOM, pp. 1–9.

  31. Chowdhury M, Rahman MR, Boutaba R (2012) Vineyard: Virtual network embedding algorithms with coordinated node and link mapping. IEEE/ACM Trans Netw 20(1):206–219

    Article  Google Scholar 

  32. Haeri S, Trajkovic L (2018) Virtual network embedding via Monte Carlo tree search. IEEE Trans Cybern 48(2):510–521

    Article  Google Scholar 

  33. Yan Z, Ge J, Wu Y, Li L, Li T (2020) Automatic virtual networkembedding: a deep reinforcement learning approach with graph convolutionalnetworks. IEEE J Sel Areas Commun 38(6):1040–1057

    Article  Google Scholar 

Download references

Funding

This work was supported in part by the State Grid Jiangxi Information & Telecommunication Company Project”Research on de-boundary security protection technology based on zero trust framework” under Grant 52183520007 V.

Author information

Authors and Affiliations

  1. School of Control and Computer Engineering, North China Electric Power University, Beijing, 102206, China

    Rixuan Qiu, Jiawen Bao & Yuancheng Li

  2. State Grid Jiangxi Information & Telecommunication Company, Nanchang, 330000, China

    Rixuan Qiu, Xin Zhou, Liang Liang & Hui Tian

  3. State Grid Jiangxi Electric Power Company, Fuzhou Power Supply Branch, Fuzhou, 344000, China

    Yanting Zeng

  4. AsiaInfo Technology (Chengdu) Co., Ltd, Chengdu, 610000, China

    Jie Shi

Authors
  1. Rixuan Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiawen Bao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuancheng Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liang Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Hui Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Yanting Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jie Shi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Author 1 (First Author): Conceptualization, Methodology, Software, Investigation, Formal Analysis, Writing–Original Draft; Author 2: Data Curation, Methodology, Formal Analysis, Writing–Original Draft; Author 3 (Corresponding Author): Conceptualization, Funding Acquisition, Resources, Supervision, Writing–Review & Editing 4: Visualization, Investigation; Author 5: Resources, Supervision; Author 6: Software, Validation Author 7: Visualization, Writing–Review & Editing Author 8: Visualization, Writing–Review & Editing Author.

Corresponding author

Correspondence to Yuancheng Li.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose. The authors have no competing interests to declare that are relevant to the content of this article. All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. The authors have no financial or proprietary interests in any material discussed in this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qiu, R., Bao, J., Li, Y. et al. Virtual network function deployment algorithm based on graph convolution deep reinforcement learning. J Supercomput 79, 6849–6870 (2023). https://doi.org/10.1007/s11227-022-04947-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04947-w

Keywords

Search

Navigation