Skip to main content
SpringerLink
Log in

A Mobility-Aware Service Function Chain Migration Strategy Based on Deep Reinforcement Learning

  • Published:
Journal of Network and Systems Management Aims and scope Submit manuscript

Abstract

With the development of network function virtualization (NFV) and mobile edge computing (MEC), service function chaining (SFC) can be deployed more flexibly on the network edge to provide users with higher quality services. However, the mobility of users might affect the quality of the perceived service and even cause service unavailability. It is thus a challenge to better migrate the service function chains to reduce prevent such degradation and improve the migration success rate. This paper investigates the SFC migration timing decision problem in user predicted movement scenarios with a known mobile path and predicted arrival time. First, we establish an prediction model of user arrival time and formulate the SFC migration process as a mathematical model. Then, we model the SFC migration process as a Markov decision process, and propose a deep Q-network based SFC migration timing decision (DQN-MTD) algorithm. DQN-MTD can be useful to perceive and predict the state changes of network resources, and select appropriate migration timing for virtual network functions (VNFs) based on SFC migration information. The experimental results show that compared with existing algorithms, DQN-MTD algorithm can reduce the average service downtime by about 14%, improve the migration success rate of SFC by about 20%, and reduce the average VNF migration time and memory when network load is low.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. ETSI: Network functions virtualisation (NFV); architectural framework, ETSI GS NFV 002 (V1.2.1). European Telecommunications Standards Institute, Sophia-Antipolis (2014)

  2. Halpern, J., Pignataro, C.: Service function chaining (SFC) architecture[M]//RFC 7665 (2015)

  3. Chen, Y.T., Liao, W.J.: Mobility-aware service function chaining in 5G wireless networks with mobile edge computing. In: ICC 2019—2019 IEEE International Conference On Communications (ICC), Shanghai, China, pp. 1–6. (2019)

  4. Zhao, D.C., Sun, G., Liao, D., Xu, S.Z., Chang, V.: Mobile-aware service function chain migration in cloud-fog computing. Future Gener. Comput. Syst. 96, 591–604 (2019). https://doi.org/10.1016/j.future.2019.02.031

    Article  Google Scholar 

  5. Zhang, F., Liu, G.M., Fu, X.M., Yahyapour, R.: A survey on virtual machine migration: Challenges, techniques and open issues. IEEE Commun. Surv. Tutor. (2018). https://doi.org/10.1109/COMST.2018.2794881

    Article  Google Scholar 

  6. Taleb, T., Ksentini, A., Frangoudis, P.A.: Follow-me cloud: When cloud services follow mobile users. IEEE Trans. Cloud Comput. 7, 369–382 (2019). https://doi.org/10.1109/TCC.2016.2525987

    Article  Google Scholar 

  7. Lu, X., Wetter, E., Bharti, N., Tatem, A.J., Bengtsson, L.: Approaching the limit of predictability in human mobility. Sci. Rep. (2013). https://doi.org/10.1038/srep02923

    Article  Google Scholar 

  8. Song, C.M., Qu, Z.H., Blumm, N., Barabasi, A.L.: Limits of predictability in human mobility. Science 327(5968), 1018–1021 (2010). https://doi.org/10.1126/science.1177170

    Article  MATH  Google Scholar 

  9. Kulkarni, V., Mahalunkar, A., Garbinato, B., Kelleher, J.D.: Examining the limits of predictability of human mobility. Entropy (2019). https://doi.org/10.3390/e21040432

    Article  Google Scholar 

  10. Gao Z., Tsiopoulos A., Friderikos V.: Dynamic VNF chains placement for mobile iot applications. In: GLOBECOM 2019—2019 IEEE global communications conference. IEEE (2019)

  11. Hu, H.F., Yang, C., Xu, L.Y., Song, T.Y.J., Dalia, B.B.: Path load adaptive migration for routing and bandwidth allocation in mobile-aware service function chain. Electron. Newsweekly (2022). https://doi.org/10.3390/electronics11010057

    Article  Google Scholar 

  12. Huang, Y.P., Chen, C., Su, Z.C., Chen, T.S., Sumalee, A., Pan, T.L., Zhong, R.X.: Bus arrival time prediction and reliability analysis: An experimental comparison of functional data analysis and Bayesian support vector regression. Appl. Soft Comput. (2021). https://doi.org/10.1016/j.asoc.2021.107663

    Article  Google Scholar 

  13. Zou, Y.J., Zhu, X.X., Zhang, Y.L., Zeng, X.S.: A space–time diurnal method for short-term freeway travel time prediction. Transp. Res. C 43, 33–49 (2014). https://doi.org/10.1016/j.trc.2013.10.007

    Article  Google Scholar 

  14. Ayoub, O., Huamani, O., Musumeci, F., Tornatore, M.: Efficient online virtual machines migration for alert-based disaster resilience. In: International conference on the design of reliable communication networks (DRCN 2019). (2019)

  15. Carpio F., Jukan A., Pries R.: Balancing the migration of virtual network functions with replications in data centers. In: IEEE/IFIP network operations and management symposium (NOMS). (2017)

  16. Eramo, V., Ammar, M., Lavacca, F.G.: Migration energy aware reconfigurations of virtual network function instances in NFV architectures. IEEE Access 5, 4927–4938 (2017). https://doi.org/10.1109/ACCESS.2017.2685437

    Article  Google Scholar 

  17. Yang, L., Yang, D.D., Cao, J.N., Sahni, Y., Xu, X.H.: QoS guaranteed resource allocation for live virtual machine migration in edge clouds. IEEE Access (2020). https://doi.org/10.1109/ACCESS.2020.2989154

    Article  Google Scholar 

  18. Qu, H., Wang, K., Zhao, J.H.: Priority-awareness VNF migration method based on deep reinforcement learning. Comput. Netw. (2022). https://doi.org/10.1016/j.comnet.2022.108866

    Article  Google Scholar 

  19. Addad, R.A., Dutra, D.L.C., Taleb, T., Flinck, H.: AI-based network-aware service function chain migration in 5G and beyond networks. IEEE Trans. Netw. Serv. Manage. 1, 472–484 (2022). https://doi.org/10.1109/TNSM.2021.3074618

    Article  Google Scholar 

  20. Liu, YJ., Feng, G., Chen, Z., Qin, S., Zhao, G.Q.: Network function migration in softwarization based networks with mobile edge computing. In: ICC 2020—2020 IEEE International Conference on Communications (ICC). IEEE (2020)

  21. Chen, R.Y., Lu, H.C., Lu, Y.J., Liu, J.X.: MSDF: A deep reinforcement learning framework for service function chain migration. In: 2020 IEEE Wireless Communications and Networking Conference (WCNC). IEEE (2020)

  22. Wang, S.Q., Urgaonkar, R., Zafer, M., He, T., Chan, K., Leung, K.K.: Dynamic service migration in mobile edge computing based on markov decision process. IEEE/ACM Trans. Netw. (2019). https://doi.org/10.1109/TNET.2019.2916577

    Article  Google Scholar 

  23. Xie, J.Z., Zhou, R., Sun, G., Sun, J., Yu, H.F.: Fast recovery for online service function chaining interruption using adaptive migration. Clust. Comput.-J. Netw. Softw. Tools Appl. (2022). https://doi.org/10.1007/s10586-021-03486-y

    Article  Google Scholar 

  24. Xu, L.Y., Hu, H.F., Liu, Y.A.: Heuristic strategy of service function chain deployment based on N-base continuous digital coding in network function virtualization environment. Electronics (2022). https://doi.org/10.3390/electronics11030331

    Article  Google Scholar 

  25. Wang, L.H., Lu, Z.M., Wen, X.M., Knopp, R., Gupta, R.: Joint optimization of service function chaining and resource allocation in network function virtualization. IEEE Access 4, 8084–8094 (2016). https://doi.org/10.1109/ACCESS.2016.2629278

    Article  Google Scholar 

  26. Liu, H.K., Jin, H., Xu, C.Z., Liao, X.F.: Performance and energy modeling for live migration of virtual machines. Clust. Comput. 16(2), 249–264 (2013). https://doi.org/10.1007/s10586-011-0194-3

    Article  Google Scholar 

  27. Pei, J.N., Hong, P.L., Xue, K.P., Li, D.F., Wei, D.S.L., Wu, F.: Two-phase virtual network function selection and chaining algorithm based on deep learning in SDN/NFV-enabled networks. IEEE J. Sel.Areas Commun. (2020). https://doi.org/10.1109/JSAC.2020.2986592

    Article  Google Scholar 

  28. Xu, L.Y., Hu, H.F., Liu, Y.: SFCSim: a network function virtualization resource allocation simulation platform. Clust. Comput (2022). https://doi.org/10.1007/s10586-022-03670-8

    Article  Google Scholar 

Download references

Funding

This work supported by National Natural Science Foundation of China (Grant Numbers 61821001 and 62090015).

Author information

Authors and Affiliations

  1. School of Information and Communications Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Hefei Hu, Wei Zhang & Panjie Qi

  2. School of Electronic Engineering, Beijing University of Posts and Telecommunications, Beijing, 100876, China

    Lingyi Xu

Authors
  1. Hefei Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lingyi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Panjie Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and method design. The experimental simulation was mainly performed by HH and WZ, assisted by LX and PQ. The first draft of the manuscript was written by WZ, reviewed by HH and LX, and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Wei Zhang.

Ethics declarations

Conflict of interest

The authors have no competing interests to declare that are relevant to the content of this article.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, H., Zhang, W., Xu, L. et al. A Mobility-Aware Service Function Chain Migration Strategy Based on Deep Reinforcement Learning. J Netw Syst Manage 31, 21 (2023). https://doi.org/10.1007/s10922-022-09713-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10922-022-09713-0

Keywords

Search

Navigation