Skip to main content
SpringerLink
Log in

DSPVR: dynamic SFC placement with VNF reuse in Fog-Cloud Computing using Deep Reinforcement Learning

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

The advent of Network Function Virtualization (NFV) has enabled the flexible provisioning of services on Fog-Cloud Computing-based Networks (CFCN) and has facilitated the implementation of 5G networks. NFV transforms hardware middleboxes into sets of software-based Virtual Network Function (VNF) that can host the growing demand for latency-sensitive services at the FCCN. Latency-sensitive and complex services can be provided by composing multiple VNF instances in the Service Function Chain (SFC). VNF instances can be deployed as virtual machines on FCCN components. In general, finding the optimal solution for placement of SFC requests based on VNF instances on FCCN is known as an NP-Hard problem. Dynamic placement of SFCs by reusing VNF instances can improve resource utilization and save time. In this paper, Dynamic SFC placement with VNF reuse (DSPVR) algorithm in FCCN using Deep Reinforcement Learning (DRL) is proposed. DSPVR is a dynamic planning model for SFC placement based on the preliminary VNFs reuse that can reconcile between Quality of Service (QoS) and service costs under FCCN constraints. DSPVR is based on DRL and has been developed with the purpose of maximizing long-term cumulative reward (LTCR). In addition, the DSPVR includes an SFC queue network for efficient distribution of VNFs required over time, which can affect the routing of future requests placement. The simulation results show the superiority of the proposed DSPVR algorithm compared to state-of-the-art methods such as DRL-SFCP and DDQP. The DSPVR outperforms the DRL-SFCP by 4.9% and 9.2% by DDQP in terms of monetary cost.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Data sharing not applicable to this manuscript as no datasets were generated or analyzed during the current study.

References

  • Abapour N, Shafiesabet A, Mahboub R (2021) A novel security based routing method using ant colony optimization algorithms and RPL protocol in the IoT networks. Int J Electr Comput Sci (IJECS) 3(1):1–9

    Google Scholar 

  • Behravesh, R., Coronado, E., Harutyunyan, D., & Riggio, R. (2019). Joint user association and VNF placement for latency sensitive applications in 5G networks. In 2019 IEEE 8th International Conference on Cloud Networking (CloudNet) Coimbra, Portugal. IEEE, pp 1–7

  • Berahmand K, Nasiri E, Li Y (2021) Spectral clustering on protein-protein interaction networks via constructing affinity matrix using attributed graph embedding. Comput Biol Med 138:104933

    Article  Google Scholar 

  • Bhamare D, Erbad A, Jain R, Zolanvari M, Samaka M (2018) Efficient virtual network function placement strategies for cloud radio access networks. Comput Commun 127:50–60

    Article  Google Scholar 

  • Chen H, Wang X, Zhao Y, Song T, Wang Y, Xu S, Li L (2018) MOSC: A method to assign the outsourcing of service function chain across multiple clouds. Comput Netw 133:166–182

    Article  Google Scholar 

  • Chenarlogh VA, Razzazi F, Mohammadyahya N (2019) A multi-view human action recognition system in limited data case using multi-stream CNN. In: 2019 5th Iranian Conference on signal processing and intelligent systems (ICSPIS), Shahrood, Iran. IEEE, pp 1–11

  • Ghai KS, Choudhury S, Yassine A (2020) Efficient algorithms to minimize the end-to-end latency of edge network function virtualization. J Ambient Intell Humaniz Comput 11(10):3963–3974

    Article  Google Scholar 

  • Ghobaei-Arani M, Shahidinejad A (2021) An efficient resource provisioning approach for analyzing cloud workloads: a metaheuristic-based clustering approach. J Supercomput 77(1):711–750

    Article  Google Scholar 

  • Gu L, Zeng D, Li W, Guo S, Zomaya AY, Jin H (2019) Intelligent VNF orchestration and flow scheduling via model-assisted deep reinforcement learning. IEEE J Sel Areas Commun 38(2):279–291

    Article  Google Scholar 

  • Jadidi A, Dizadji MR (2021) Node clustering in binary asymmetric stochastic block model with noisy label attributes via SDP. In: 2021 International Conference on smart applications, communications and networking (SmartNets), Glasgow, United Kingdom. IEEE, pp 1–6

  • Kaur K, Mangat V, Kumar K (2020) A comprehensive survey of service function chain provisioning approaches in SDN and NFV architecture. Comput Sci Rev 38:100298

    Article  Google Scholar 

  • Khoshkholghi MA, Gokan Khan M, Alizadeh Noghani K, Taheri J, Bhamare D, Kassler A, Yang X (2020) Service function chain placement for joint cost and latency optimization. Mob Netw Appl 25(6):2191–2205

    Article  Google Scholar 

  • Kouah R, Alleg A, Laraba A, Ahmed T (2018) Energy-aware placement for IoT-service function chain. In: 2018 IEEE 23rd International Workshop on computer aided modeling and design of communication links and networks (CAMAD), Barcelona, Spain. IEEE, pp 1–7

  • Li D, Hong P, Xue K (2018) Virtual network function placement considering resource optimization and SFC requests in cloud datacenter. IEEE Trans Parallel Distrib Syst 29(7):1664–1677

    Article  Google Scholar 

  • Li G, Feng B, Zhou H, Zhang Y, Sood K, Yu S (2020) Adaptive service function chaining mappings in 5G using deep Q-learning. Comput Commun 152:305–315

    Article  Google Scholar 

  • Liu Y, Lu H, Li X, Zhang Y, Xi L, Zhao D (2020) Dynamic service function chain orchestration for NFV/MEC-enabled IoT networks: a deep reinforcement learning approach. IEEE Internet Things J 8(9):7450–7465

    Article  Google Scholar 

  • Liu C, Wang J, Zhou L, Rezaeipanah A (2022) Solving the multi-objective problem of IoT service placement in fog computing using cuckoo search algorithm. Neural Process Lett. https://doi.org/10.1007/s11063-021-10708-2

    Article  Google Scholar 

  • Mao W, Wang L, Zhao J, Xu Y (2020) Online fault-tolerant VNF chain placement: A deep reinforcement learning approach. In: 2020 IFIP networking conference (networking), Paris, France. IEEE, pp 163–171

  • Mostafavi S, Hakami V, Sanaei M (2021) Quality of service provisioning in network function virtualization: a survey. Computing 103(5):917–991

    Article  Google Scholar 

  • Movahhed Neya N, Saberi S, Rezaie B (2022) Design of an adaptive controller to capture maximum power from a variable speed wind turbine system without any prior knowledge of system parameters. Trans Inst Meas Control 44(3):609–619

    Article  Google Scholar 

  • Nasiri E, Berahmand K, Rostami M, Dabiri M (2021) A novel link prediction algorithm for protein-protein interaction networks by attributed graph embedding. Comput Biol Med 137:104772

    Article  Google Scholar 

  • Orlowski S, Wessäly R, Pióro M, Tomaszewski A (2010) SNDlib 1.0—Survivable network design library. Netw Int J 55(3):276–286

    Google Scholar 

  • Pei J, Hong P, Xue K, Li D (2018) Efficiently embedding service function chains with dynamic virtual network function placement in geo-distributed cloud system. IEEE Trans Parallel Distrib Syst 30(10):2179–2192

    Article  Google Scholar 

  • Ponmagal RS, Karthick S, Dhiyanesh B, Balakrishnan S, Venkatachalam K (2021) Optimized virtual network function provisioning technique for mobile edge cloud computing. J Ambient Intell Humaniz Comput 12(6):5807–5815

    Article  Google Scholar 

  • Raayatpanah MA, Weise T (2018) Virtual network function placement for service function chaining with minimum energy consumption. In: 2018 IEEE international conference on computer and communication engineering technology (CCET), Beijing, China. IEEE, pp 198–202

  • Rezaeipanah A, Mojarad M, Fakhari A (2022) Providing a new approach to increase fault tolerance in cloud computing using fuzzy logic. Int J Comput Appl 44(2):139–147

    Google Scholar 

  • Shahidinejad A, Ghobaei-Arani M, Esmaeili L (2020) An elastic controller using Colored Petri Nets in cloud computing environment. Clust Comput 23(2):1045–1071

    Article  Google Scholar 

  • Shiri A, Khosroshahi GK (2019) An FPGA implementation of singular value decomposition. In: 2019 27th Iranian conference on electrical engineering (ICEE), Yazd, Iran. IEEE, pp 416–422

  • Solozabal R, Ceberio J, Sanchoyerto A, Zabala L, Blanco B, Liberal F (2019) Virtual network function placement optimization with deep reinforcement learning. IEEE J Sel Areas Commun 38(2):292–303

    Article  Google Scholar 

  • Song X, Zhang X, Yu S, Jiao S, Xu Z (2017) Resource-efficient virtual network function placement in operator networks. In: GLOBECOM 2017–2017 IEEE global communications conference, Singapore. IEEE, pp 1–7

  • Subramanya T, Harutyunyan D, Riggio R (2020) Machine learning-driven service function chain placement and scaling in MEC-enabled 5G networks. Comput Netw 166:106980

    Article  Google Scholar 

  • Sun J, Huang G, Sun G, Yu H, Sangaiah AK, Chang V (2018) A Q-learning-based approach for deploying dynamic service function chains. Symmetry 10(11):646

    Article  Google Scholar 

  • Tofighy S, Rahmanian AA, Ghobaei-Arani M (2018) An ensemble CPU load prediction algorithm using a Bayesian information criterion and smooth filters in a cloud computing environment. Softw Pract Exp 48(12):2257–2277

    Article  Google Scholar 

  • Trik M, Pour Mozaffari S, Bidgoli AM (2021) Providing an adaptive routing along with a hybrid selection strategy to increase efficiency in NoC-based neuromorphic systems. Comput Intell Neurosci 2021:1–8

  • Trik M, Molk AMNG, Ghasemi F, Pouryeganeh P (2022) A hybrid selection strategy based on traffic analysis for improving performance in networks on chip. J Sens 2022:1–19

  • Wang L, Mao W, Zhao J, Xu Y (2021a) DDQP: a double deep Q-learning approach to online fault-tolerant SFC placement. IEEE Trans Netw Serv Manag 18(1):118–132

    Article  Google Scholar 

  • Wang T, Fan Q, Li X, Zhang X, Xiong Q, Fu S, Gao M (2021b) DRL-SFCP: adaptive service function chains placement with deep reinforcement learning. In: ICC 2021b-IEEE international conference on communications, Montreal, QC, Canada. IEEE, pp 1–6

  • Xiao Y, Zhang Q, Liu F, Wang J, Zhao M, Zhang Z, Zhang J (2019) NFVdeep: adaptive online service function chain deployment with deep reinforcement learning. In: Proceedings of the international symposium on quality of service, NY, USA, pp 1–10

  • Xie Y, Wang S, Dai Y (2020) Revenue-maximizing virtualized network function chain placement in dynamic environment. Futur Gener Comput Syst 108:650–661

    Article  Google Scholar 

  • Xu Z, Zhang X, Yu S, Zhang J (2018) Energy-efficient virtual network function placement in telecom networks. In: 2018 IEEE international conference on communications (ICC), Kansas City, MO, US. IEEE, pp 1–7

  • Yang S, Li F, Trajanovski S, Chen X, Wang Y, Fu X (2019) Delay-aware virtual network function placement and routing in edge clouds. IEEE Trans Mob Comput 20(2):445–459

    Article  Google Scholar 

  • Yue Y, Cheng B, Liu X (2020) Resource optimization and delay-aware virtual network function placement for mapping SFC requests in NFV-enabled Networks. In: 2020 IEEE 13th international conference on cloud computing (CLOUD), Beijing, China. IEEE, pp 267–274

  • Zeng M, Fang W, Zhu Z (2016) Orchestrating tree-type VNF forwarding graphs in inter-DC elastic optical networks. J Lightwave Technol 34(14):3330–3341

    Article  Google Scholar 

  • Zhang S, Jia W, Tang Z, Lou J, Zhao W (2022) Efficient instance reuse approach for service function chain placement in mobile edge computing. Comput Netw 211:109010

    Article  Google Scholar 

  • Zheng J, Tian C, Dai H, Ma Q, Zhang W, Chen G, Zhang G (2019) Optimizing NFV chain deployment in software-defined cellular core. IEEE J Sel Areas Commun 38(2):248–262

    Article  Google Scholar 

Download references

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Maybod Branch, Islamic Azad University, Maybod, Iran

    Fatemeh Zahedi, Mohammadreza Mollahoseini Ardakani & Ahmad Heidary-Sharifabad

Authors
  1. Fatemeh Zahedi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammadreza Mollahoseini Ardakani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ahmad Heidary-Sharifabad
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammadreza Mollahoseini Ardakani.

Ethics declarations

Conflict of interest

We certify that there is no actual or potential conflict of interest in relation to this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zahedi, F., Mollahoseini Ardakani, M. & Heidary-Sharifabad, A. DSPVR: dynamic SFC placement with VNF reuse in Fog-Cloud Computing using Deep Reinforcement Learning. J Ambient Intell Human Comput 14, 3981–3994 (2023). https://doi.org/10.1007/s12652-022-04465-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-022-04465-w

Keywords

Search

Navigation