Deep Reinforcement Learning for VNF Placement and Chaining of Cloud Network Services

Abstract

With Network Function Virtualization (NFV), network services can be swiftly and efficiently constructed by instantiating Virtual Network Functions (VNFs) on host servers. In order to optimize resource utilization (such as bandwidth) and meet Service-Level Agreements (end-to-end delay for instance), efficient solutions are required for VNF placement, consolidation, and chaining. To tackle these challenges, we propose two approaches: (1) a novel Formal Concept Analysis (FCA)-based method focused on placing VNFs on appropriate virtual machines, and (2) a Deep Reinforcement Learning-based approach that integrates two parallel modules for VNF placement and chaining: Markov Decision Process (MDP) and Long Short-Term Memory (LSTM).

The parallel operation of these modules enables the extraction and capture of the current NFV environment and historical transitions. In our proposal, we employ Policy Gradient for agent training, aiming to identify suitable hosts for each VNF in the Service Function Chain (SFC), thereby enhancing various quality metrics such as latency, resource cost, and throughput. Simulation results show the effectiveness of our approach, achieving a 47% improvement in rewards compared to the deep VNF approach and a 52% improvement compared to the First Fit approach.